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SECTICN  1.0 
INTRlX)UCTION 


1 . 1  PURPOSE 

This  document  describes  the  Phase  I  SBIR  study  titled  "Pfethodology 
Development  For  Verification  and  Validation  Of  Flic^it  Critical  Systans 
Software",  Contract  No.  F33615-89-C-3610.  This  work  was  sponsored  by  the 
Wri^t  Research  and  Develcpnent  Center,  Fli^t  Dynamics  Laboratory,  Fli^t 
Control  Division  at  Wright  Patterson  Air  Force  Base.  The  purpxjse  of  this 
effort  was  to  develcp  an  approach  for  an  innovative,  integrated  verification 
and  validation  methodology  which  focuses  on  hi^ily  coupled  flight  critical 
systems  software. 

1.2  SCOPE 

Current  trends  in  ^plications  of  advanced  control  and  integration 
technologies  are  bringing  about  the  development  of  on-board  systems  that  are 
designed  to  enhance  combat  effectiveness  and  survivability  in  ever  increasing 
hostile  canbat  environments.  Flight  critical  systens  (including  integrated 
fli^t  and  propulsion  control,  integrated  flight  and  ire  control, 
self-repairing  flight  control,  vehicle  management,  pilot-vehicle  interface,  and 
flight  vehicle  sensors)  are  being  controlled  and  integrated  in  software.  These 
developments  are  putting  an  ever  increasing  load  on  the  developiient, 
verification  and  validation  (V&V)  of  flight  critical  systenns  software.  This 
Phase  I  SBIR  effort  has  researched  and  developed  an  innovative  approach  for 
advancing  the  state  of  the  art  in  the  application  of  verification  and 
validation  methodology.  This  methodology  can  be  developed  into  a  Conputer 
Aided  Verification  and  Validation  Ervgineering  Systen  (CAV^ES)  hosted  in  a 
workstation  environment.  FTI's  cpproach  provides  a  quantum  improvement  in  the 
effectiveness  and  efficiency  of  the  software  environment  (containing  methods, 
disciplines,  documentation,  tools  and  controls)  needed  for  verification  and 
validation  of  highly  reliable,  fault  tolerant  flight  critical  systems  software. 
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The  CAV^ES  environment  offers  many  advantages  over  current  V&V  methods 
and  procedures.  These  advantages  include: 

o  A  hi^ily  avitonated  syston  that  speeds  the  V&V  process  by  approximately 
5  to  1,  saving  valuable  time  and  moriey. 

o  A  workstation  environment  viiich  provides  the  user  with  a  powerful 
conputational  environment  which  can  utilize  many  of  the  more 
sophisticated  PCS  design  and  analysis  tools  available. 

o  An  environment  easily  adaptable  to  futuristic  Air  Force  systems  such  as 
hypersonic  air  vehicle,  control  of  unmanned  aerial  vehicles,  and  even 
battle  management  systans. 

o  The  c^jability  to  evaluate  designs  and  design  excursions  within  the  V&V 
environment  and  transport  the  evaluations  to  other  phases  of  V&V 
efforts. 

o  Provides  a  CASE  type  working  environment  for  transportation  and 

traceability  of  V&V  requirements,  design,  and  test  analysis  data. 

o  Is  oriented  specifically  to  performing  software  V&V,  but  is  applicable 
to  development  efforts. 

o  Provides  a  User  Help  Guide  to  specific  V&V  steps  to  be  performed. 

o  Provides  the  Cc^>ability  to  perform  r^id  prototyping  through  its  tool 
and  interface  environment. 

o  Can  accept  irputs  fron  many  external  systems:  (eg., MEAD,  TAE,  ...) 

through  cotmon  interfaces  defined  for  transporting  external  inputs. 

o  Can  be  used  to  aid  in  providing  a  flight  critical  systems  engineering 
function  throucfiout  FCS  software  development  cycle. 

The  unique  features  of  the  Conputer  Aided  Verification  and  Validation 
Engineering  System  makes  it  a  very  cost  effective  environment  to  be  used  by  the 
flight  critical  systons  analyst  to  perform  the  V&V  process.  These  features 
include: 

o  An  integrated  V&V  tool  set  usable  through  all  development  phases. 

o  User  friendly  interfaces;  point /click  mouse,  pull  down  windows,  quick 
views  and  transfer  of  data/conparisons,  common  functionality  between 
tools,  command  driven  and  menu  driven  human  interface. 


2 


o 


Provides  hooks  for  adding  future  tools  and  a  means  for  interfacing  and 
driving  real-time  simulations, 
o  Offers  an  environment  aiding  automation  of  real-time  testing, 
o  Usable  by  developers,  V&V  organizations,  and  research  groi;ps. 
o  Is  an  evolutionary  system  vAiicdi  accommodates  new  tools  to  meet  the 
growing  user  requirements. 

1 . 3  Document  Organization 

This  report  contains  the  following  sections: 
o  Section  1  is  the  Introduction. 

o  Section  2  is  an  Executive  Summary  of  the  approach  taken  in  this  effort 
and  the  results  of  each  task. 

o  Section  3,  Technical  Discussions,  contains  all  of  the  details  of  the 
work  performed  and  presents  the  Ccnputer  Aided  Verification  and 
Validation  Engineering  System  cono^Jtvial  design. 
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SECTION  2.0 
EXECUTIVE  SUMMARY 


2.1  PURPOSE  OF  THE  WORK 

The  applications  of  digital  technology  to  Flight  Critical  Systems  (FCS)  has 
allowed  far  reaching  advances  in  air  vi^iicles  throu^  vehicle  optimization  of 
the  air  vehicle  controllability,  performance,  safety,  and  mission 
effectiveness.  Flight  critical  systems  (including  integrated  flight  and 
propulsion  control,  integrated  flight  and  fire  control,  self-repairing  flight 
control,  vehicle  management,  pilot-vehicle  interface,  and  flight  vehicle 
sensors)  are  being  controlled  and  integrated  through  software.  The  spectrum  of 
flight  control  technology  has  e:q>anded  beyond  the  classically  recognized  role 
of  stability  and  control  and  flying  qualities  and  now  includes  technologies 
covering  functional,  physical,  and  pilot-vehicle  interfaces  vhich  inpact 
aircraft  design  options,  combat  effectivemss,  and  survivability. 

As  shown  in  Table  2.1-1,  there  are  a  nunber  of  prdblen  areas  in  the 
verification  and  validation  of  flight  critical  syston  software.  First, 
inplementation  of  most  flight  critical  systems  is  being  performed  through 
software.  This  means  tliat  an  increasing  share  of  the  development  effort  and 
costs  will  continue  to  be  driven  by  software.  Thus,  there  is  a  very  iirportant 
demand  for  a  well  disciplined,  efficient  software  development,  verification, 
and  validation  methodology.  This  study  specifically  addresses  verification  and 
validation  methodology,  but  discusses  tools  and  techniques  which  are  equally 
applicable  during  development. 

Second,  not  only  cure  these  systems  being  developed  with  software  being  a 
primary  iitplenenting  factor,  but  also  software  has  become  the  means  by  which 
these  flight  critical  systens  are  integrated.  Thus,  a  well  disciplined, 
efficient  software  V&V  methodology  is  required,  more  now  than  ever  before  to 
both  guide  the  development  and  conduct  V&V  of  highly  integrated  FCS  software. 
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TABLE  2.1-1  SOFTWARE  PROBLEM  AREAS  IN 
VERIFICATION  AND  VALIDATION  OF  FLIGHT 
CRITICAL  SYSTEMS  SOFTWARE 


EXPANDED  ROLE  OF  SOFTWARE  IN  FLIGHT  CRITICAL  SYSTEMS 
REQUIRES  A  WELL  DISCIPLINED  SOFTWARE  V&V  METHODOLOGY 


INTEGRATION  OF  CRITICAL  FLIGHT  SYSTEMS  THROUGH  SOFTWARE 
REQUIRES  A  RELIABLE  V&y  PROCESS  TO  ASSURE  THAT  SAFETY 
OF  FLIGHT  REQUIREMENTS  ARE  MET 


•  SHIFT  TO  HOLS  REQUIRES  DEVELOPMENT/UPDATE  OF 

-  SOFTWARE  TOOLS 

-  IMPLEMENTATION  ENVIRONMENT 

-  VERIFICATION  AND  VALIDATION  TECHNIQUES 


FREQUENT  CHANGES  IN  SQFTWARE  DEVELOPMENT  &  DOCUMENTATION 
STANDARDS  REQUIRES  FLEXIBLE  SOFTWARE  DEVELOPMENT  AND  V&V 
APPROACHES  THAT  CAN  BE  UPDATED  EFFICIENTLY 


•  INCREASED  COMPLEXITY  OF  INTEGRATED  FLIGHT  CRITICAL  SYSTEMS 
INCREASES  THE  MAGNITUDE/COST  OF  SOFTWARE  DEVELOPMENT 
AND  V&V  EFFORTS 


V8  vn 


Third,  the  shift  fron  assetibly  language  and  certain  higher-order  languages 
(i.e.,  JOVIAL,  etc.)  to  Ada  as  the  accepted  language  for  flight  critical 
systems  is  in  process  and  is  inpacting  current  and  future  software  development 
in  flight  critical  systems.  Efforts  are  under  way  to  inprove  the  performance 
of  the  software  inplementation  technology  used  within  Ada  conpilers  to  meet 
flight  critical  systenvs  requirements.  These  inprovsnents,  vhen  achieved,  will 
enable  the  cost  effective  features  of  Ada  to  be  used  and  more  effectively 
provide  fault  tolerant  code  vdrLch  is  essential  for  flight  critical  systems. 
However,  during  this  transition  period,  development  of  software  tools,  choice 
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of  different  inplesmentation  strategies  and  conversion  from  previous 
inplenentation  languages  to  Ada's  inpletentation  structure  and  discipline  will 
put  additional  burdens  in  software  development  and  V&V  disciplines. 

Fourth,  changes  in  government  software  development  standards  effect 
required  documentation  levels,  milestone  reporting,  etc.,  have  also  added  to 
the  need  of  develc^ing  a  disciplined  software  development  and  V&V  methodology. 
Previously  used  military  standards  applicable  to  software  development 
(MIL-STD-483  and  MIL-S'ID-490)  have  been  amended  by  MIL-STD-2167A. 
MIL-STD-2167A  has  a  formidable  list  of  documentation  requironents  on  software 
vhich  are  hipacting  software  develcpment  efforts  on  current  flight  critical 
systens.  Accountability  to  these  software  standards  and  tailoring  of  these 
standards  to  specific  applications  is  yet  another  area  vMch  must  be  factored 
into  a  software  development  and  V&V  methodology. 

Finally,  the  evolvenent  of  hic^y  integrated  analog/digital  and  digital/ 
digital  flight  critical  systens  vduch  are  mechanized  to  provide  redundancy  and 
the  capability  to  reconfigure  or  provide  graceful  degradation  with  failure 
insertions  has  conplicated  the  verification  testing  of  such  systans.  Coupling 
these  mechanizations  with  interfaces  to  sensors  avionic  systems,  pilot 
interfaces,  and  AI  oriented  vehicle  management  systems  has  made  the  cost  of 
software  development  one  of  the  major  drivers  in  syston  life  cycle  cost.  In 
this  area  alone  a  disciplined  software  verification  and  validation  methodology 
addressing  all  aspects  of  critical  fli^t  software  development  can  reap 
tremendous  savings. 

2.2  OVEPVIEW  OF  TECHNICAL  APPROACH 

The  technical  approach  to  the  development  of  a  methodology  for  verification 
and  validation  of  flic^i-t  critical  systems  software  was  carried  out  in  a  three 
step  process.  The  first  step.  Task  1,  was  to  perform  a  requirements  analysis 
to  identify  specific  nfieds  to  be  met  in  V&V  of  fli^t  critical  systems 
software.  The  second  task  performed  in  conjunction  with  the  first  was  a  data 


6 


collection  effort.  The  third  step  (Task  3)  was  the  synthesis  of  a  FCS  software 
V&V  methodology  which  would  meet  the  requirements  defined  in  the  reqi’irenents 
analysis.  Task  1.  This  process  for  the  Phase  I  effort  is  shown  in  Figure 
2.2-1. 
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Figure  2.2-1  Overview  of  Program  Approach 


2.2.1  Task  1  -  Reouirgnents  Analysis.  The  purpose  of  the  requironents 
analysis  task  was  to  define  a  conplete  set  of  requirements  whic±i  must  be  met  in 
the  V&V  of  flight  critical  systems  software.  The  caiple-xity  of  flight  critical 
systems  development  has  outpaced  the  management  and  technical  resources 
supporting  their  acquisition.  The  rising  conplexity  in  hardware  technology. 
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software  technology/  and  the  integration  of  systens  stresses  the  capability  to 
design,  build,  and  test  such  systems.  An  initial  draft  of  the  requirements 
definition  was  prepared  primarily  from  Frontier's  experience  in  previous 
software  V&V  efforts.  This  draft  provided  ^  guide  for  areas  of  investigation 
during  the  Data  Collection  task.  The  information  gained  from  the  data 
collection  task  was  then  in  turn  used  to  ipdate  the  requirements  definition 
resulting  in  a  corrplete  and  consistent  set  of  requirements. 

2.2.2  Task  2  -  Data  Collection.  The  approach  taken  in  the  data  collection 
task  was  to  first  develop  checklists  (see  Appendix)  to  help  organize  the 
acquisition  of  information  on  each  element/function  required  in  the  software 
verification  and  validation  methodology.  The  checklists  addressed  specifics 
about  available  softwaire  analysis  tools  and  techniques  available  and  current 
percept  ions /experiences  in  the  development  of  FCS  softwaire.  Included  within 
these  checklists  was  a  list  of  ^^plicable  Government  standards  which  must  be 
met  in  the  development  of  flight  critical  systems.  Examples  of  such  standards 
include  MIL-STD-483, -490,  MIL-STD-2167A,  MIL-STO-1521,  ANSI/MIL-STD-1815A-Ada, 
MIL-F-9490,  MIL-STD-8457,  to  name  a  few.  One  of  the  problems  besetting 
software  developers  today  is  being  knowledgeable  in  the  requirenents  spelled 
out  in  many  standards  and  even  more  importantly,  being  able  to  meet  the  intent 
of  the  standards  in  a  cost  effective  manner. 

Next,  a  catalog  of  applicable  software  tools  used  in  all  phases  of 
development  and  verification  and  validation  of  flight  critical  systems  software 
was  prepared.  This  included  control  law  analysis  tooJs,  software  development 
tool  sets,  documentation  aids  and  tools,  configuration  management  software 
tools,  and  verification  and  validation  software  tools.  As  a  starting  point 
this  list  was  conpiled  from  our  knowledge  and  hands-on  experience  with  many  of 
the  tools.  This  list  was  then  expanded  through  the  information  gained  in 
literature  searches,  siorveys  of  software  tools  used  by  flight  critical  system 
developers,  and  focused  interviews  of  key  government  and  industry  experts. 
This  list  of  tools  was  entered  into  our  Tools  Data  Base  Management  System  for 
ease  of  reference  and  quick  recall. 
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The  next  step  in  the  data  collection  process  was  to  conduct  in  depth 
interviews  with  key  government  and  industry  ej^rts  in  the  development  of 
flight  critical  systems.  These  interviews  included  personnel  at  Wricfit 
Research  and  Development  Center  —  Fli^t  Dynamics  Laboratory,  Air  Force  Flight 
Test  Center,  NASA  Dryden  Fli^t  Research  Facility,  McAir,  General  Dynamics, 
Rockwell  International,  Honeywell,  Softech,  and  High  Plains. 

The  prepared  checklist  was  used  to  guide  the  discussions  on  FCS  software 
verification  and  validation  tools  and  techniques.  At  the  ccnoletion  of  these 
interviews,  the  raw  data  findings  were  prepared  and  provided  to  WRDC/FIGX.  As 
part  of  the  data  collection  tcisk,  a  Data  Base  Management  System  (DBMS)  was 
developed  to  help  organize  and  record  the  many  sources  of  data  and  the  tools 
and  techniques  identified.  This  DBMS  was  created  on  DBASE  III+  and  hosted  on 
an  IBM  catpatible  PC.  New  data  is  continuing  to  be  added  to  this  system  and  it 
will  serve  as  a  reference  source  for  proposed  Phase  II  efforts. 

2.2.3  Task  3  -  Verification  and  Validation  Concept  Design.  The  third  task  in 
our  technical  approach  to  development  of  a  V&V  methodology  was  to  synthesize  a 
preliminary  design  in  sufficient  detail  to  demonstrate  that  the  design  is 
technically  sound  and  that  V&V  functions  have  been  defined  to  address  the  FCS 
software  V&V  requironents .  A  variety  of  tools  and  techniques  were  considered. 
The  approach  synthesized  provides  a  user-friendly  host  environment  for  current 
and  future  V&V  tools  applications. 

The  scope  of  tools  and  techniques  will  allow  the  user  to  assess  development 
of  critical  flight  systems  software  from  systems  requirements  definition  on 
through  "iron-bird"  validation  testing  leading  to  flight  test.  Features  which 
are  included  in  the  design  are: 
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o 


Verification  cind  validation  activities  performed  in  a  series  of  steps 
v?hidh  are  interfaced  throu^  the  customer  and  coordinated  with  the 
develc^xent  phases  of  the  fli^t  critical  systems  software, 
o  Utilization  of  control  systens  analysis  tools  to  verify  flight 

critical  systens  software  in  a  manner  which  will  address  safety  of 
flic^t  issues  and  provide  a  high  level  of  confidence  in  the  e>^cted 
performance  of  the  systen  design  and  inplenentation. 
o  Use  of  both  syston  level  and  statement  level  emulations  for 

verification  of  flight  critical  systems  software  design.  System  level 
emulation  will  be  used  for  analysis  of  performance,  stability,  and 
redundancy  management.  Statenent  level  onulation  will  be  used  to 
exercise  the  code  itself. 

o  Structuring  a  verification  and  validation  approach  to  provide  an 

appropriate  balance  between  external  V&V  ivities  and  developer  V&V 
efforts. 

o  The  use  of  proven  Independent  Verification  and  Validation  (IV&V) 

techniques  for  evaluation  of  the  prime  contractor's  development 
process  and  software  design. 

o  Use  of  advanced  hierarchial  modeling  techniques  for  the  development  of 
topological  network  trees  vhich  are  more  easily  understood  than  code 
representations,  and  vhich  constitute  a  deliverable  data  base  to  serve 
as  an  effective  tool  for  software  change  inpact  analysis, 
o  Analysis  techniques  which  will  provide  an  assessment  of  overall  system 
performance  as  well  as  verification  of  system/software  design  and 
coding. 

o  Use  of  a  structured  data  base  of  proven  validation  test  procedures 
vhich  have  been  used  to  perform  "iron-bird"  validation  testing  of 
highly  integrated  flight  control  systems.  This  data  base  addresses 
tests  of  performance,  redundancy  managonent,  mechanization,  failure 
mode  and  effects,  aircraft  systems  and  interfaces,  and  all  other 
matters  effecting  flight  performance  and  safety  of  flight, 
o  Utilization  of  quick-look  analysis  tools  applicable  to  ground  based 
simulation  and  flight  test  data  and  application  of  advanced  data 
reduction  techniques. 
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A  FCS  software  V&V  methodology  has  been  designed  vhich  provides  these 
features  and  is  discussed  in  Section  3.5.  Specific  techniques  and  tools  are 
presented  to  address  each  of  the  phases  involved  in  an  V&V  of  flight  critical 
systems  software. 

2.3  SUMMARY  OF  THE  RESULTS 

The  results  of  this  conoept\:ial  design  study  have  shown  that  there  is  a 
growing  need  to  develop  a  methodology  by  \Aiich  flight  critical  systems  software 
can  be  verified  and  validated  for  performance  and  safety  inpacts.  The  nature 
of  the  evolving  technology  and  its  application  to  FCS  software  verification  and 
validation  requirements  has  current  V&V  methods  lagging  behind  design  methods 
and  tools.  It  is  recognized  throuc^out  government  and  industry  that  FCS 
software  V&V  requires  knowledgeable  and  skilled  individuals  utilizing  proper 
tools  and  techniques  to  successfully  ccnplete  the  V&V  effort  in  a  timely 
manner.  This  report  provides  an  overview  of  the  development  process  of  flight 
critical  systems  and  the  roles  of  verification  and  validation  vMch  go 
hand-in-hand  with  the  develcpnent  process.  It  provides  a  conceptual  design  for 
a  conputer  aided  environment  to  perform  FCS  software  verification  and 
validation. 


2.3.1  Results  of  FCS  Software  V&V  Requirements  Definition  and  Data  Collection 

The  process  of  defining  V&V  requiretents  is  iterative  and  was  performed  in 
conjunction  with  our  data  collection  effort.  Initial  draft  requirenents  were 
prepared  and  tpdated  as  new  information  was  gained  frcxn  our  interviews  with  key 
government  and  industry  experts  and  through  data  gained  in  literature  reviews. 
Key  to  the  definition  of  these  requirements  was  having  a  sound  understanding  of 
current  FCS  software  development  and  V&V  practices.  Figure  2.3-1  is  a 
representation  of  the  development  process  with  specifications  of  verification 
and  validation  levels. 

A  variety  of  accepted  techniques  are  currently  used  to  perform  the  V&V  of 
FCS  software.  These  techniques  are  supplemented  with  tools  (manual  procedures 
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or  software  programs)  to  aid  in  the  analysis  tasks  and  the  bookkeeping  of  the 
results.  Since  the  V&V  of  EX3S  software  is  normally  performed  as  a  parallel 
effort  with  the  development  efforts#  accepted  techniques  used  in  the  V&V 
efforts  are  driven  by  the  timing  availability  of  FCS  software  development 
products.  The  current  V&V  tasks,  allied  techniques,  and  V&V  objectives  are 
sumnarized  in  Table  2.3-1. 


Figure  2.3-1  PCS  Development  and  V&V  Phases 
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TABLE  2.3-1  V  &  V  REQUIREMENTS 


OBJEETIVE/PURPOSE  APPLICABLE  TECHNIQUES /TOOLS 


Systera  ^Decification 
Verificatiws 

Evaluated  to  ensure  tliat  system/ 
subsystem  considered  will 
fulfill  mission  goals  and 
objectives. 

o  Pequirements  Analysis 
o  DocLsnentation  Peview 

Control  Law  Analysis 

Assure  control  algorithms 
adequacy.  Verify  equation 
accuracy;  evaluate  functional 
relationshipis  and  functional 
p)erfotmanoe  (tljiiing,  sequencing, 
etc.) 

o  Pequirements  Analysis 
o  Control  Law  Analysis 
o  E^Tulation/Sinulatlon 

Evaluation  of 

Development  Planning 

Evaluate  for  satisfactory 
standards  t  practices, 
schediles,  planning,  controls, 
reviews,  audits,  CM  change 
control,  prdbleni  resolution,  VtV 

o  Peview  Fbnagement  Plan 
o  CPC**  Peview 

Evaluation  of 

Software  Development 

Methodology 

Preventive  Measure.  Sound 
design,  coding,  and  test 
techniques  reduce  raiitjer  of 
errors  made  during  development. 

o  Dooment  Peview 

-  Standards 

-  Plans 

-  Configuration  Management 
Provisions 

Software  Feqairements 
Verification 

Requirements  evaluated  for 
adequacy,  coopleteness, 
accuracy,  testability,  and 
traceability  to  higher  level 
^jeclf ications . 

o  Pequirements  Analysis 
o  Critical  Pequirements 

Ident 1 f icati on 
o  Docunentation  Peview 

Software  Design 

Evaluate  development  products 
to  ensure  tcdmical  viability 
and  contribute  to  refinement 
paooess.  Eiisure  software  design 
represents  a  clear,  consistent 
and  accurate  translation  of 
software  requirements. 

o  Design  Analysis 
o  Performance  Analysis 
o  Document  Review 
o  Top  Down  Progi  amning 
o  System  Level  Emulation 
o  Consistency  Checker 
o  Standardlration 

Code  Correctness 

Test  and  evzduate  developers 
code  using  Indgxaident  tools. 

Code  Is  checked  for  errors, 
amissions  and  incorrect  trans¬ 
lations.  E;valv3ate  logic,  file 
structuring,  execution  paths  and 
limitations.  Interfaces,  etc . 
hbchine  level  open-loop  tests 
aiKl  unit  and  module;  closed-loop 
at  subsystem/system.  Examine 
timing. 

o  Code  Analysis 
o  Comparator 
o  Corpilcr 
o  Interface  Checker 
o  Doernent  Review 
o  Cross  Referenoo 
o  Cross  Assentoler 
o  SimJatlon 
o  Instruction  Trace 

Identify  unejqpected  paths  for 
information  flow  through  a  prx?- 
gram  by  analyzing  the  clues 
characteristic  of  sneak  paths  in 
netvotk  trees/ flow  graphs. 

o  Sneak  Analysis 

Software  Validation 

o  Development  Testr 
o  System  Tests 

o  Flight  Tests 

o  Control  Lane 

Pesponse 

o  Kandlinq  Ojantitief^ 
o  Functional  Tests 

Determine  irfiether  all  software 
and  system  performanese,  inter¬ 
face,  functional  and  test 
requirements  are  fulfilled. 

-  Bv^iry  requircjnenf  in 
adequately  tested 

-  All  subsystems  an?  pix^x'ily 
int  ogialed 

-  All  system  respaixscs  are 
atirxiUiile  for  pTfi'imvio'  ,uv1 
safet  y . 

o  Test  Plan/Procjedure  Peview 
o  Test  Case  Generation 
o  liot -Bench  Simulatoi 
o  Muinfram:'  Shrulation 
n  IrriEi  Pil'd  SLiulation 
o  Airrraft  Flight 

o  Be^iundancy 

Kin-igement 

o  Faijurv  KiivuymrMit 

im,  tlut  PM/FM  system  r 

fi'f  ujii  nfT^'nt  ft)i  st 

t'f  la.lun'; 
|wu.uiw'trir  i  ‘>  o! 

|vit  jvit  . 

V'  .‘'y.'-.t  m  ior‘. 

<  Inm  IMitI  1  .It  ion 

}>?T 

‘lit 

t'.iin?  ‘'•rLiii'  >' 

a  1 

aid  in  jti’r  V-  «  f  V\V 

.V  ■  *-  1  .r.>  ■  . 
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As  can  be  seen,  a  large  number  of  techniques  have  been  developed  to  aid  in  the 
verification  and  validation  of  softvrare.  A  deliverable  Data  Base  Managonent 
Systan  (DBMS)  was  iirplemented  by  FTI  to  help  organize  and  record  the  many 
sources  of  data  and  the  tools  and  techniques  identified  in  this  effort. 

Elvolvanent  of  Conputer  Aided  Software  Engineering  (CASE)  tools  continues 
towards  providing  software  develcpment  with  the  environment  of  an  integrated 
tool  set  including  planning,  analysis,  design,  documentation,  static  analysis, 
prototyping,  dynamic  analysis,  simulation,  and  construction  of  executable 
systons.  Unfortunately,  CASE  tools  have  not  reached  the  point  where  this  broad 
application  of  tcisks  has  been  integrated  into  one  development  environment. 
Similarly,  the  use  of  CASE  tools  in  a  V&V  environment  offers  great  premise  and 
sonne  CASE  tools  have  been  specifically  designed  to  perform  reverse  engineering 
on  existing  designs,  a  necessary  V&V  cap>ability. 

The  near  term  trends  in  flight  control  systems  (FLCS)  are  expjanding  the  use 
of  real-time,  on-board  cptimization  and  intelligent  controls  to  achieve  high 
performance  and  provide  for  damage  tolerance  and  self-healing  designs.  These 
near  term  ETCS  already  are  addressing  the  inner-loop,  outer-loop,  and 
redundancy  management  functions.  EX::S  integration  has  an  even  more  challenging 
inpact  on  software.  Forecasts  and  projections  for  PCS  in  the  21st  century 
indicate  a  number  of  significant  inrpacts  on  FCS  software  V&V  as  indicated 
below. 

o  Significant  increases  in  conputer  pxjwer  will  cause  major  expansion  in 
scope  and  character  of  onboard  systems. 

o  Development  of  circhitectural  branches  within  redundant  systems  will 
add  verification  and  validation  conplexity,  onbedded  replicated  or 
dissimilar  subchannels  for  self  monitoring  could  reduce  redundancy 
management  complexities  at  higher  rates. 

o  Increased  throu^iput  and  energing  new  architectures  are  allowing 
sensor  fusion  with  information  integration  and  display,  requiring 
expianded  FCS  verification  and  validation  roles. 
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o  Trends  towards  systems  hii^y  integrated  through  FLCS  —  because  of 

mission  and  performance  benefits  —  leads  to  more  testing  at  system 
levels,  interdisciplinary  expertise,  and  pilot  involvenent. 

o  Increase  of  control  effectors  and  reduction  in  actuator  redundancy 

levels  for  self  repair/reconfigurable  flight  control  increases  the 
complexity  of  Vcilidation  testing. 

o  Decision-Aiding  systans  in  a  real-time  environment  require  validation 
of  knowledge  bases  vAiich  currently  have  no  accepted  validation 
methods. 

The  use  of  hic^r  order  languages  (HOLs)  eases  the  task  of  FCS  design 
process.  HOLs  aillow  the  flight  control  engineer  to  more  easily  follow  the 
design  through  iirplenentation  in  software.  Current  problems  with  Ada  (tasking 
and  rendeavDus)  do  not  stop  its  use;  the  specific  prdblati  areas  can  be 
avoided.  However,  there  is  still  the  question  that  once  code  is  recompiled,  it 
is  difficult  to  say  that  new  code  is  good  versus  assenbly  language  patches 
approach. 

FCS  designers  are  moving  towards  providing  control  law  block  diagrams  from 
which  code  can  be  generated  automatically.  GE's  program  called  FASTER  directly 
generates  1750A  asserbly  code.  Many  of  the  linear  analysis  tools  (Ctrl-C, 
Matrix  X,  MATLAB,  etc.)  purport  to  generate  code  directly  from  block  diagrams 
which  can  then  be  lised  directly  in  simulati.ons  to  test  from  design  through 
simulation.  Tools  of  these  types  eliminate  many  of  the  coding  errors  vMch  can 
occur  during  the  process  of  changing  a  flight  critical  design  to  code. 

Future  FCS  systems  will  still  have  to  address  interfacing  with  existing, 
older  systems.  There  is  a  wide  generation  of  conputers  currently  fielded  and 
this  will  always  be  the  case.  Many  of  the  older  computers  cannot  support  HOL. 
Development  contractors  are  mo'/ing  towards  using  RISC  conputers.  However, 
currently  there  is  not  adequate  svpport  tools  in  this  environment. 

Transportable  software  is  also  being  addressed.  However,  software 
conpilers  are  currently  a  problem  here  because  there  is  no  agreed  to  standard. 
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Also,  timing  is  one  of  the  most  critical  elenents  in  flight  critical  software 
and  this  effects  transportability.  Coimon  nvodule  approches  requires  that  the 
developer  look  at  real  needs  in  V&V.  Resources  are  currently  being  spent  in 
designing  software  test  stations/tools  that  will  test  to  requirements  and  using 
language  translators  to  inplement  new  front -ends  to  these  test  stations/tools. 
Control  law  filters  have  alreacty  been  transported.  Use  of  Ada  will  help 
transportability  in  future  developments. 

Vehicle  Managanent  Systems  (VMS)  is  the  new  focus  in  flight  critical 
systems.  However,  the  designers  must  be  realistic  about  what  they  propose  and 
use.  It  is  cost  prohibitive  and  not  even  possible  to  test  all  combinations 
required  to  "adequately"  validate  a  highly  integrated  PCS. 

Redundancy  &  Monitoring  (R&M)  is  another  design  area  which  drives  V&V 

requirements.  From  a  design  point,  coverage  of  all  failure  mechanisms  is  the 

prcblen  here.  There  is  the  question  of  quad  vs  triplex  systems.  Triplex 

—7 

systems  can  meet  the  1  x  10  requirement,  but  it  is  difficult  to  meet  an 
imposed  requirement  of  fail-op,  fail-cp  without  going  to  a  quad  system.  From  a 
cost  and  development  viewpoint,  a  quad  voter  runs  twice  as  long  as  a  triplex 
voter;  software  conplexities  at  least  double  for  every  channel  added  due  to 
combinatorial  considerations . 

A  number  of  lessons  learned  can  be  gained  from  past  efforts  in  development 
of  fli<^t  critical  systems  software. 

o  In  general,  prc±>lems  arise  in  specifications  across  on-boctrd  aircraft 
systems.  Understanding  and  documentation  of  interfaces  between  systems 
are  often  lacking. 

o  The  use  of  simulation  for  testing  integrated  systens  is  questionable.  This 
is  particularly  true  in  the  area  of  sensors.  The  PCS  testing  is  dependent 
on  models  for  high  technology  sensors  and  in  this  area  modeling  is  very 
difficult.  Systems  integration  adds  more  ccxrbinations  of  conditions  vhich 
need  to  be  tested. 

o  People  vho  have  tested  systems  have  to  put  information  gained  back  into  the 
loop.  The  problems  that  were  encountered  and  how  they  were  solved  is  often 
not  reported. 
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o  One  very  large  requirenent  is  requirements  and  specifications  for  control 
laws.  There  is  a  lack  of  a  reasonable  MIL-Spec  for  flight  control;  PIO 
prediction  is  an  exanple  of  this.  Mil  Prime  Standard  8785-C  is  not 
adequate;  it  is  a  back-\jp  guide. 

o  Most  errors  are  in  design.  These  are  generally  found  in  systens 

integration  testing.  B-2  put  a  lot  of  time  and  money  to  get  set  up  for 
systems  integration  testing  and  that  has  paid  off  v?ell. 

2.3.2  Conceptual  Design  for  a  V&V  Nfethodoloov  for  PCS  Software 

The  FTI  technical  ^proach  to  the  development  of  the  flight  critical 

systeniis  verification  and  validation  methodology  is  based  on  a  balanced 
allocation  of  technical  skills/  proven  V&V  tools  and  techniques,  and  evolving 
software  developmentcil  test  methodologies.  The  inplenentation  of  oiar 

methodology  can  prpvide  a  workstation  environment  providing  the  needed  tools 
and  techniques  for  verification  and  validation  of  fli^t  critical  systems. 

Proven  V&V  tools  vAiich  can  be  used  directly  to  meet  the  V&V  requirements 
will  be  reviewed  as  candidates  to  be  used  in  the  development  of  a  Coiputer 

Aided  Verification  And  Validation  Engineering  System  (CAV^ES) .  The  CAV^ES 
will  be  hosted  in  a  workstation  environment  and  will  provide  the  user  with 
ready  access  to  those  requirements,  design,  and  development  details  needed  to 
assess  the  state  of  development  of  EX^S  software.  Much  of  the  early  development 
verification  activities  will  utilize  V&V  tools  hosted  in  the  workstation 
environment  itself  and  will  provide  analysis  data  and  results  vhich  can  be 
carried  from  one  V&V  stage  to  the  next.  The  V&V  methodology  will  also  address 
the  validation  test  activities  vhich  take  place  when  the  FCS  software 
development  progresses  to  the  point  vhere  testing  of  software  and  subsystems 
utilizes  hotbench  simulators,  simulators  requiring  mainframes  for  conputitional 
support  and  finally  vhen  testing  moves  into  an  ironbird  test  environment. 

The  CAV^ES  will  provide  an  environment  in  which  the  flight  control 
engineer  or  software  engineer  can  quickly  and  easily  access  and  analyze  design 
information,  software  code,  or  generate  data  to  verify  and  validate  ti:S  soft¬ 
ware.  It  will  allow  the  engineers  to  deal  with  all  phases  of  the  development 
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cycle  and  tackle  the  prc±)lan  of  maintaining  a  continuity  of 
requiriQnents/design  evaluations  across  development  i^iases. 

The  Ccnputer  Meted  Verification  and  Validation  Engineering  System  (see 
Figure  2. 3. 2-1)  vdll  provide  the  following  functions:  V&V  Executive  (VEXEC) , 
Tool  Interface  tfenager  (TIM) ,  User  Interface  (UI) ,  Simulation  Ccnputer  Inter¬ 
face  (SCI) ,  Data  Recording  System  (DRS) ,  Data  Base  Manager  (DBM) ,  Autonatic 
Pilot  Functions  (APF) ,  and  the  Data  Analysis  System  (DAS) . 


Figure  2. 3. 2-1  Ccnputer  Mded  Verification  and  Validation 
Engineering  System  (CAV^ES) 
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To  aid  the  FCS  software  analyst  in  performing  V&V  tasks,  CAV^ES  retains 
its  own  data  base  and  librairy  of  tools.  The  data  base  is  structured  to  be  able 
to  store  and  retrieve  the  data  items  which  are  a  product  of  executing  the 
verification  and  validation  analysis  tools.  The  tool  library  consists  of  two 
parts,  a  generic  tools  set  and  an  external  tools  set.  The  generic  tool  set  is 
a  group  of  V&V  tools  which  will  perform  basic  V&V  functions  on  FCS  software. 
The  external  tools  set  represents  user  selected  or  new  tools  which  need  to  be 
interfaced  through  the  Tool  Interface  Manager. 

The  Tool  Interface  Manager  (TIM)  selects  v^ch  interfaces  are  used  with  the 
selected  tools  so  that  the  output  of  the  library  tool  is  put  into  a  standard 
format  acceptable  to  the  Data  Base  ^fenager.  If  a  new  tool  (external  tool)  is 
to  be  interfaced  to  CAV^ES,  the  TIM  has  an  interface  build  capability  which 
aids  the  user  in  building  a  functional  interface  which  converts  data  to  a 
format  consistent  with  the  existing  data  base. 

The  V&V  Executive  (VEXEC)  monitors  and  coordinates  the  operation  of 
CAV^ES  functions.  The  VEXEC  execution  involves  issuing  cortmands  to  the  Tool 
Interface  f-tinager,  tlie  Simulation  Ccxtputer  Interface,  the  Data  Recording 
System,  the  Data  Base  Manager,  the  Automatic  Pilot  Functions  and  the  Data 
Analysis  System.  It  receives  commands  and  sends  responses  to  the  user  via  the 
User  Interface. 

The  purpose  of  the  User  Interface  (UI)  is  to  provide  direct  access  to  the 
various  software  tools  functionalities,  while  relieving  the  user  from  needing 
to  be  intimately  knowledgeable  about  the  software  tools  as  stand-alone  systems 
and  adapting  to  their  various  styles  and  syntax.  This  ireans  that  a  user  who 
wants  to  obtain  a  time  history  plot  of  data  generated  by  a  simulation  tool, 
CTRL-C  or  MATRIX-X  for  exanple,  does  not  need  to  Joiow  the  particular  commands 
for  the  tool  package  for  simulation  and  plotting.  However,  the  UI  does  not 
confine  the  experienced  tool  user  to  stay  within  the  UI  interface,  but  provides 
a  direct  tool  mode  in  vrfiich  the  user  can  execute  tool  commands  within  the 

9 

CAV^ES  environment  to  perform  any  simulation  and  plotting  activity  allowed  by 
the  tool.  The  UI  is  built  in  a  windows  environment  to  provide  quick  expansion 
or  contraction  of  backup  information,  aiding  in  the  verification  and  validation 
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process.  The  functionality  of  a  tool  can  be  accessed  via  point-and-click  mouse 
operations  on  icons,  menu,  and  form  driven  screens. 

The  Simulation  Conputer  Interface  (SCI)  is  used  to  coiinunicate  to  the 
simulation  corputers.  Communication  may  take  place  over  serial  lines  to 
various  devices,  over  ethemet,  and  over  direct  bus  links.  The  user  may  open  a 
terminal  window  for  each  of  these  connections  and  manually  type  commands.  All 
ccaimands,  along  with  responses,  will  be  logged  and  sent  to  the  DBM  to  be 
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recorded  in  a  test  execution  log.  Other  subsystems  of  CAV^ES  may  also  send 
ccstinands  to  the  simulation  ccnputers.  All  ccximands  indicate  if  a  response  is 
expected.  SCI  will  then  pass  along  the  command  and  wait  for  the  response,  if 
necessary . 

The  Data  Recording  System  (DRS)  is  responsible  for  recording  simulation 
data  (both  real-time  and  non-reaLl-time)  and  transferring  data  to  the  data  base 
manager.  The  DRS  receives  its  commands  from  the  VEXEC.  Before  a  test  begins, 
the  VEXEC  sends  a  list  of  commands  to  be  executed  (recording  script) .  The 
real-time  simulation  recording  takes  place  via  a  link  (bus  link  or  ethemet) 
connected  to  the  simulation  computers  via  SCI.  Proper  synchronization  is 
critical  if  a  valid  set  of  data  is  to  be  recorded. 

The  Data  Base  Manager  (DBM)  serves  two  piorposes.  First,  to  create  and 
maintain  data  base  files  and  second,  to  conduct  data  transactions  for  other 
CAV^ES  subsystans.  The  DBM  is  conposed  of  two  processes  to  serve  these 
pxarposes:  Vexec  Interface  and  Build  Ipdate.  The  interface  process  conducts 
transactions  while  Build  Update  is  used  to  create  and  maintain  data  base 
files.  To  assist  in  performing  these  processes,  a  commercially  available  data 
base  managCTicnt  system,  UNIFY,  will  be  used. 

The  Automatic  Pilot  Functions  (APF)  subsystem  provides  the  capaibility  for 
the  CAV^ES  systan  to  send  pilot  coftmands  to  the  aircraft  flight  control 
system.  The  APF  provides  the  capability  to  provide  flight  test  functions  (FTF) 
for  testing  of  performance  peurameters  and  to  fly  fundamental  maneuvers,  VEXEC 
obtains  commanded  maneuvers  from  the  test  procedures  via  the  Data  E^se 
Manager.  The  APF  sends  these  ccxTmands  to  the  simulation  cc*Tputers  and  flight 
control  system  via  the  SCI.  The  APF  uses  a  transportable  auto-pilot  model 
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which  may  be  hosted  on  the  simulation  corputers  (mainframes)  or  in  the  CAV  ES 
workstation  environment,  dependent  on  the  particular  type  of  comtunications 
link  used  between  the  CAV^ES  and  the  simulation  conputers.  This  provides 
flexibility  in  the  choice  of  this  camiunication  link  frcan  one  inplenentation  to 
the  next. 

The  Data  Analysis  Systan  (DAS)  is  designed  to  provide  the  validation 
engineer  the  capability  to  examine  the  data  recorded  during  a  test  and  to 
perform  data  reduction  techniques  on  the  recorded  data.  The  recorded  data  can 
be  displayed  both  on  a  CRT  and  on  a  hardcopy  device.  Data  displayed  during 
simulation  execution  will  generally  be  plots  of  variables  as  a  function  of  time 
and  mode  switches.  Post  test  data  analysis  can  be  performed  providing 
performance  parameters  of  flic^t  critical  systens. 

2.3.3  CAV^ES  FCS  V&V  Capabilities 

Capabilities  to  be  included  within  the  CAV^ES  environment  will  provide 
the  flight  control/software  engineers  the  capability  to  assess  the  FCS  software 
design  in  terms  of  performance,  stability,  and  redundancy  management  analysis. 
It  will  provide  both  static  and  dynamic  code  analysis  tools  for  verification 
and  validation  of  flight  criticcil  systems  code.  It  will  also  provide  the 
capability  to  perform  quick- look  analysis  of  generated  data.  It  will  provide 
tie  means  to  verify  and  validate  FCS  software  through  control  of  real-time 
simulators  driven  by  proven  test  procedures /test  cases. 

Aircraft  Flight  Critical  Systems  Analysis.  The  CAV^ES  will  provide  the 
capability  to  evaluate  the  adequacy  of  the  control  laws  with  respect  to 
performance  and  stability  and  to  evaluate  system  mechanization  with  respect  to 
redundancy  managatient,  timing  and  bus  loading.  To  perform  these  analysis,  3 
general  types  of  flif^t  control  analysis  tools  will  be  used:  a  linear  arialysis 
and  design  tool,  a  nonlinear  simulation  tool,  and  a  system  level  emulator. 
Ccjmercially  available  linear  analysis  and  design  tools  provide  a  corprehensive 
interactive  control  design  and  analysis  software  language  system  including 
state-of-the-art  primitives  in  classical  and  modem  control  synthesis,  matrix 
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analysis,  dynamic  systan  analysis,  parameter  estimation,  and  graphical 
presentation. 


A  general  purpose  non-linear  aircraft  simulation  program  incorporates 
specific  aircraft  characteristic  throuc^i  user-defined  modules  for  the 
aerodynamic  forces  and  moments,  the  propulsion  systan,  the  control  system, 
etc.  It  can  be  run  to  trim  the  aircraft  for  any  desired  flight  condition,  to 
generate  linear  state  models  for  the  trinmed  flight  condition,  and  to  generate 
time  history  responses  for  user-defined  inputs  representing  conmands  or 
external  disturbances. 

A  system  level  enulator  can  be  used  to  analyze  the  correctness  of  module 
logic  and  functions,  bus  loading,  timing  and  redundancy  management,  as  well  as 
analyzing  the  operational  capabilities  of  a  system  and  its  conformance  to 
system  requirements. 

Software  Design  Verification.  The  objective  of  software  design  is  to 
confirm  the  technical  adequacy  of  the  desii^  Much  of  this  effort  involves 
manual  review  of  requironents  and  design  .specifications,  requiring  methodical 
and  diligent  attention  in  matching  requironents  to  design  and  in  evaluating 
designs.  It  is  in  this  arv-a  vhere  the  application  of  CASE  tools  can  provide 
benefits  in  verification.  In  general,  CASE  tools  leverage  the  requirements 
analysis  and  design  specification  phases  of  the  software  development  cycle 
while  more  traditional  tools  are  more  applicable  in  the  software  inplenentation 
phase. 

FCS  Software  Code  Verification.  Specific  static  analysis  techniques  vhich 
will  be  initially  implonented  will  be  selected  as  a  part  of  the  Phase  II 
effort.  However,  capabilities  to  perform  software  sneak  analysis  and  to 
perform  instruction  level  onulator  code  execution  are  exartples  of  static  and 
dynamic  anaJysis  tools,  respectively,  vhich  provide  a  strong  code  verification 
tool  base  and  are  planned  for  inplementation  in  CAV^ES. 

Stand  Alone.  Dynamic  Subsystem,  and  Integrated  System  V&V.  The  engineering 
and  formal  system  and  software  testing  utilizes  a  bottom-up  philosophy. 


Testing  starts  with  a  lowest  level  code  (unit  code)  and  progresses  upward  to 
system-level  testing.  Verification  of  results  is  performed  at  each  level 
before  progressing  to  the  next  level.  Test  plans  and  procedures  are  prepared 
for  each  level  of  formal  testing  and  test  results  are  documented.  Problems  or 
discrepancies  detected  during  any  phase  of  testing  are  documented, 
investigated,  and  acted  upon. 

CAV^ES  will  be  used  to  communicate  to  simulation  conputers  and  subsystems 
to  drive  individual  subsysten  tests,  integrated  subsyst^i  tests,  and  finally 
system  level  tests.  The  Maneuvering-simulation  Validation  Autonation  (MVA) 
system  currently  under  envelopment  by  FTI  for  SAAB-SGANIA  will  provide  just 
this  type  of  capability.  The  design  structure  of  CAV^ES  will  incorporate 
many  of  the  features  in  the  MVA  system. 

The  CAV^ES  is  an  integrated  V&V  tool  set  to  be  used  throughout  FCS 
software  development  phases.  It's  integration  and  application  with  the  FCS 
development  phases  is  shown  in  Figure  2. 3. 3-1.  It  is  usable  by  developers,  V&V 
organizations,  and  research  groups.  Its  workstation  provides  the  user  with  a 
powerful  conputational  environment  which  can  utilize  many  of  the  more 
sophisticated  FCS  design  and  analysis  tools.  CAV^ES  provides  a  CASE  type 
working  environment,  user  friendly  interfaces,  and  provides  hooks  for  addition 
of  future  tools.  It  provides  a  means  for  interfacing  and  driving  real-time 
simulations  and  offers  an  environment  aiding  automation  of  real-time  testing. 
In  short,  the  CAV^ES  is  an  evolutioncuy  system  which  accommodates  new  tools 
to  meet  the  growing  user  requiranents  in  verification  and  validation  of  fliglit 
critical  systems  software. 
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2.4  POTENTIAL  APPLIO^ICNS  OF  THE  CAV^ES  DEVELOPMENT  EFFORT 

The  Cotputer  Aided  Verification  and  Validation  Engineering  System  has  many 
ccxtmercial  applications  vdiich  will  bring  large  savings  to  its  users.  A  great 
deal  of  interest  has  been  ejjpressed  by  government  agencies  and  industry  leaders 
in  providing  new  capabilities  in  the  verification  and  validation  of  the 
evolving  fli^t  critical  systans.  The  CAV^ES  will  be  directly  usable  by 
industry  developers  of  systems  v^ch  cire  flight  critical  and  are  driven  by 
software.  It  will  also  be  usable  by  manufacturers  of  automated  systems 
controlled  by  software  vAiich  can  effect  the  performance  of  their  product  and 
the  safety  of  their  user. 

The  aircraft  manufacturers  including  General  Dynamics,  MsAir,  Rockwell, 
Lockheed,  Northrop,  and  Grumman  all  currently  are  faced  with  the  probl^is  of 
developing  and  integrating  fli^t  critical  systems  as  their  major  product 
output.  They  have  expended  a  large  amount  of  resources  in  atteirpting  to  build 
and  maintain  verification  and  validation  capabilities  to  keep  pace  with  the 
increasing  growth  in  the  use  of  software  in  virtually  all  of  their 
applications.  The  CAV^ES  offers  these  industry  leaders  an  inexpensive  tool 
vrtiich  can  be  easily  implonented  and  custcmized  to  meet  their  specific  needs. 
Of  course,  the  commercial  market  extends  beyond  the  United  States  market  and 
covers  our  NATO  allies  and  sipported  neutral  countries  such  as  Sweden. 
Coipanies  such  British  Aerospace  (BAE) ,  Ptesserschmitt-Boelkow-Blohm  (MBB) , 
SAAB-SCANIA,  and  (3X1  Avionics  cire  all  heavily  involved  in  the  same  pursuit  of 
developing,  verifying,  and  validating  fli^t  critical  systens. 

CAV^ES  will  be  of  use  to  the  many  subcontractors  to  the  prime  aircraft 
manufacturers.  These  include:  the  developers  of  flight  control  systems  such  as 
Honeywell,  General  Electric,  and  Lear  Siegler;  engine  manufactures  such  Pratt 
&  Whitney,  General  Electric,  and  Rolls  Royce;  and  other  manufactures  of 
subsystems  such  as  hydraulic  actuators,  brciking  systems,  and  the  large  number 
of  integrated  avionics  coiponents  going  into  modem  day  aircraft.  All  of  these 
developers  of  subsystems  face  the  problem  of  verification  and  validation  of 
software  used  to  control  and  interface  their  subsystems  with  othei  flight 
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critical  systons.  This  requires  the  verification  activities  involved  in  the 
early  phases  of  design  and  development  of  software,  and  also  requires  that  they 
provide  and  a  realistic  environment  with  which  to  test  their  subsystems, 
validating  that  they  meet  specified  performance  goals  and  safety  requirements. 

The  interest  and  cqpplication  of  a  CAV^ES  in  the  government  is  readily 
apparent  by  sitiply  reviewing  the  many  procurement  and  research  activities  being 
announced  on  a  daily  basis.  Exanples  of  these  include  solicitation  by:  NASA 
Ames  Research  Center  (SBIR:  03.10  -  "Development,  Testing  and  Verification  Of 
Flight  Critical  Systems") ;  DARPA  (SBIR:  90-087  -  "Low  Cost  Reconfigurable 
Generic  Corputer  Workstations  for  Simulation  Research/Development/Analysis")  ; 
Air  Force  (SBIR:  A90-470  -  Verification  and  Validation  of  Expert  Systems) ;  and 
a  recent  procurement  announced  by  the  Avionics  laboratory  titled  "Advanced 
Avionics  Verification  And  Validation".  The  CAV^ES  has  capabilities  which  are 
directly  applicable  to  each  of  these  and  offers  the  flexibility  to  expand  to 
future  required  capabilities.  The  direct  use  of  the  CAV^ES  by  the  many 
research  groups  in  the  government  organizations  is  not  limited  to  WRDC  but 
extends  to  all  government  services  engaged  in  the  research  and  development  and 
testing  of  flight  critical  systens  software. 

CAV^ES  offers  a  capability  to  the  independent  software  development  and 
software  verification  and  validation  contractors  who  the  government  solicits  to 
parform  IV&V  activities  on  flight  critical  systexB.  For  contractors  such  as 

o 

these,  the  CAV  ES  offers  a  low  cost  test  bed  tool  which  can  be  applied  at  all 
phases  of  verification  and  validation.  CAV^ES  is  a  tool  which  the  SPOs  can 
offer  or  specify  as  a  support  tool  with  the  )cnowledge  that  it  is  a  tool  vdrLch 
can  produce  the  type  of  products  that  are  needed  to  validate  PCSs  safety 
requirCTvents . 

Aside  fran  the  contractors  who  build  flight  critical  systems,  the  CAV^ES 
offers  capabilities  of  interest  to  ccmmercial  manufactures  of  systens  which  use 
software  that  can  effect  the  user's  safety.  The  car  manufactures  have  braking 
system,  engine  systems,  and  ride  quality  systems  which  are  controlled  by 
software  and  could  inpact  safety  if  failure  occurs.  Developers  of  robotic 
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controlled  manufacturing  processes  adso  need  a  capability  to  develop,  verify, 
and  validate  software  v^ch  is  safe  to  use  in  a  manufacturing  environment. 

O 

In  sijramary,  the  usefulness  and  numbers  of  applications  of  the  CAV'^ES  is 
very  large  including;  cill  branches  of  the  militciry  government  agencies 
supporting  the  development  of  flic^t  critical  systems;  the  aircraft 
manufacturers  and  their  many  subcontractors;  the  companies  vdio  provide 
independent  and  consulting  sipport  in  the  verification  and  validation  of  fli^t 
critical  systons;  and  other  conmercial  manufacturers  v^o  use  software  as  an 
autCHTBting  and  integrating  means  in  the  development  of  their  commercial 
products.  The  resourc3es  saved  on  future  FCS  programs  are  many  times  the 
CAV^ES  development  costs. 
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SECTION  3.0 
TEX3iNICMi  DISOJSSIOJS 


3.1  DE:VE^JOP^ENT  PHASES  FCR  ElilGHT  CRITICAL  SYSTElyB 

3.1.1  Overview 

The  develc^xnent  process  of  criticad  systems  has  many  variations  in 

literature,  but  the  basic  states  aire  universal.  We  choose  to  follow  that 
defined  in  Reference  1.  The  stages  are: 

o  Study  phase 

o  Concept  phase 

o  System  requirements  phaise 

0  Systen  design  {^lase 

o  Subsysten  requirements  i*ase 

o  Subsystem  design  fhase 

o  SW  requirements  phase - HW  specification  phase 

o  Basic  software  design  phase 

o  Detailed  software  design  phase 

o  Coding/module  test  phase - ^HW  development  phase 

o  SW  integration  on  host  conputer  phase 

o  SW  integration  on  target  ccxrputer  phase - HW  integration  phase 

o  Subsystem  integration  phase 

o  System  integration  phase 

o  Flic^it  test  phase 

o  Production  phase 

o  Postdevelopment  svpport  or  in-service  phase 
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CXir  main  eiphasis  in  this  stucty  starts  at  the  syston  requirenents  phase  and 
carries  throu^  system  integration  tp  to  flight  test.  Before  enbarking  on  a 
discussion  of  the  develcpnent  process,  it  is  helpful  to  see  how  the  roles  of 
verification  and  validation  go  hand-in-hand  with  the  development  process. 
Figure  3.1-1  is  a  representation  of  the  development  process  with  specifications 
of  verification  and  validation  levels.  Starting  at  the  bottan  of  the  figure, 
verification  may  be  defined  ais  the  demonstration  that  each  step  in  the 
design/development  process  (left  side  of  Figure  3.1-1)  is  correct  and  that  the 
software  program  is  a  correct  rendition  of  the  software  design.  Validation, 
shown  in  the  middle  of  the  figure,  may  be  defined  as  the  demonstration  of  the 


Figure  3.1-1  FCS  DE7\/ELOPMENT  AND  V&V  PHASES 
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correct  perfomance  of  the  entire  subsystem/systen.  With  this  brief 
introduction  to  V&V/  we  will  return  to  a  discussion  of  the  presented 
development  phases. 


3.1.2  Systems  Pequiretnents/Desicm  Phase 

The  systen  requirements/design  f^oase  produces  a  Systan  Specification 
containing  customer  and  other  known  requirements  viiich  must  be  later  validated 
during  the  system  integration  efforts.  In  addition  to  the  Systen 
Specification,  a  System  ^fechanization/Architect^are  document  is  normally 
produced  describing  the  entire  systen  structure,  connections  between 
subsystems,  preliminary  data,  and  design  of  the  systen.  These  documents  then 
form  the  bases  for  requirements  and  design  of  subsystems. 

3.1.3  Subsystem  Reouirements/Design  Phase 

During  the  subsystem  requirements/design  phase,  each  subsysten  will 
normally  have  a  Subsystem  Requirements/Design  Specification  containing: 

o  A  functional  description  of  the  subsystem  including  a  copy  of  the 
specific  subsystem  functional  requirements  contained  in  the  System 
Specification 

o  The  subsysten  design  concept  and  subsystem  architecture 

o  Subsystem  criticality  classification 

o  Requirenents  on; 

Sensors  and  effectors 

Processors 

Displays  and  controls 

o  A  detailed  descaription  of  the  subsystem's  interface  with  all  other 
subsystems  and  with  the  system  as  a  vrfiole.  Each  subsysten  will 
generate  requirements  vrtiich  will  be  implemented  in  software  or 
hardware.  Detailed  requirements  for  the  hardware  and  software 
components  to  be  used  to  iitplenent  the  subsystens  will  be  derived  from 
the  functional  requironents  during  this  phase.  Seme  requirements  for 
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the  hardware  cxiqponents  may  be  determined  by  use  of  C3ertain  software 
cxarponents  and  by  the  design  environment, 
o  The  cxciputing  power  (speed  and  memory)  required  to  execute  the 

software  within  the  time  allowed. 

o  The  availability  of  a  particular  software  develc^ment  environment. 

Also,  certain  functions  might  be  required  that  can  realized  only 
through  equipment  specific  software, 
o  Built-In  Test  (BIT) . 
o  Data  format  conversation. 

o  Specific  sensor  functions  (e.g..  Inertial  Navigator,  Display) . 

3.1.4  Software  Reouirenripnt  a /Hardware  Specification  Phase 

Sepairation  for  subsystem  functions  between  hardware  and  software  ccnponents 
is  Ccirried  out  for  each  subsystem  duririg  the  Software  Requirements/Hardware 
Specification  Phase.  Precise  boundaries  between  hardware  and  software  cannot 
be  determined  in  an  effective  way  prior  to  corpletion  of  the  Subsystem 
Requirements  and  Subsystem  Design  Phases.  Results  of  the  Software  Requiranents 
and  Hardware  Specification  Phase  are  normally  recorded  in: 

o  Software  Requirements  Documents  (including  descriptions  and 
derivations  of  all  algorithms  to  be  iirplemented) 
o  Equipment  Specifications 

o  An  Interface  Definition  and  Control  Document 

3.1.5  Basic  Software  Design  Phase 

The  Basic  Software  Design  Phase  transforms  the  software  requirarvents  for  a 
sxhsystan  into  a  software  module  design  and  a  subsysten  test  design.  Program 
structure,  program  flow,  module  functions,  and  interfaces  with  other  modules 
are  determined  during  this  phase.  Timing  considerations  are  normally  addressed 
within  a  framing  concept  defining  the  module  calling  sequences  based  on 
experience  from  existing  similar  systems.  Bus  load  analysis  should  be 
confirmed  by  rapid  prototyping  tests. 
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Results  of  the  preliminary  software  design  process  should  be  contained  in  a 
Software  Design  Specifications  and  a  Software  Test  Requirements  Document. 
These  documents  are  verified  agciinst  the  appropriate  Software  Requirenents 
Document  during  formal  review  at  the  end  of  the  Software  Preliminary  Design 
Phase. 

3.1.6  Detailed  Software  Design  Phase 

During  the  Detailed  Software  Design  Phase,  the  internal  module  structure, 
the  algorithms  to  be  inplanented,  and  the  data  structures  to  be  used  are 
cotpletely  defined.  The  internal  module  structure  is  described  in  functional 
flows  or  pseudo-code.  Also,  module  test  procedures  are  prepared. 

The  following  information  developed  during  the  Detailed  Software  Design 
Phase  is  added  to  the  Software  Design  Specification. 

o  Module  descriptions 

o  Detailed  func^  \  -  jil  flows 

o  Detailed  d' L'  i.lows 

o  Detailed  interface  and  data  description 

o  Module  test  specifications. 

At  the  end  of  the  Detaiiled  Software  Design  Phase,  a  Critical  Design  Review 
is  held  to  confirm  that; 

o  The  design  is  conplete.  All  internal  modules  are  present.  All 
interfaces  are  defined. 

o  All  software  requirements  have  been  satisfied, 
o  The  required  software  quality  attributes  are  achieved. 

3.1.7  Module  Codina/Test  Phase 

The  Software  Design  Document  and  Software  Test  Document  form  the  basis  for 
module  coding  and  test  activities,  respectively.  During  the  Module  Coding/Test 
Phase,  the  modules  cure  coded  and  commented;  syntax  errors  are  debugged,  and 
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ccxie  inspection  and  code  wailk  throu^is  are  performed.  All  previously  specified 
module  tests  are  prepared.  Test  stubs  and  drivers  are  established.  Static  and 
dynamic  module  tests  are  executed.  Detected  errors  are  corrected. 

The  Software  Design  document  is  updated  to  be  reflective  of  the  inplemented 
design  and  a  listing  of  source  caxie  is  added  to  the  documentation.  The 
Software  Test  Document  procedures  are  ccnpleted  to  form  a  module  test  report. 

3.1.8  Software  and  Software/Hardware  Intecrration  Phase 

During  this  fiiase,  the  process  is  to  integrate  the  individual  modules  into 
a  working  subsystem.  This  is  normally  acccnplished  on  a  host  development 
corputer.  Irputs  and  outputs  to  each,  module  are  simulated  and  the  modules  are 
successively  integrated  one  at  a  time  into  a  software  subsystem.  When  all 
modules  are  integrated/  the  software  subsystem  is  moved  to  a  target  ccnputer 
and  the  code  is  retested.  Software/Hardware  test  procedures  are  filled  in  and 
bound  to  provide  a  Software/Hardware  Integration  Test  Report. 

3.1.9  Subsystem  Intecrration  Phase 

At  this  point  the  subsystem  code  is  moved  to  the  target  corputer  (flight 
corputer) .  The  software  corponents  that  handle  the  hardware  interfaces  to  the 
fli^t  conputer  are  integrated  one  at  a  time  and  tested.  Finally,  the 
Subsyston  Test  Procedures  are  executed  on  the  flight  corputer  and  a  test  report 
of  results  is  prepeued. 

3.1.10  System  Integration  Phase 

The  System  Integration  Phase  is  performed  in  a  similar  manner  to  the 
subsyston  integration.  Subsystems  are  integrated  individually  into  the  system 
and  tested.  The  end  task  of  the  syston  integration  is  to  validate  that  the 
overall  syston  satisfies  the  system  requirements  in  the  Syston  Specifications. 
The  system  tests  performed  at  this  phase  are  normally  very  extensive  and  cover 
all  of  the  functional  and  performance  capabilities  to  be  achieved  at  the 
different  points  in  the  flight  envelope.  When  the  system  has  been  adequately 
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validated  for  system  safety  properties,  and  handling  qualities  and  performance, 
the  system  is  moved  into  flight  test. 

3.1.11  Flight  CriticcLL  System  Engineering 

One  function  which  is  particularly  important  in  systen  testing,  but  extends 
back  to  the  beginning  of  the  system  design  is  that  of  flight  critical  systems 
engineering.  With  the  growth  in  integration  of  flight  critical  systems,  there  is 
a  growing  need  to  provide  a  flight  critical  systans  engineering  function  to 
administer  to  the  needs  of  the  flight  critical  portion  of  the  aircraft.  The 
functions  vhich  fall  into  EX^  engineering  can  be  summarized  as  follows: 

o  Allocate  ciircraft  requironents  to  Vehicle  Management  Systan  (VIO  flight 
criticcil  requirements.  This  includes  definition  of  architecture, 
redundancy  levels,  fault  coverage  requiranents,  functional  partitioning, 
iteration  rates,  systan  time  delays,  information  flow,  through-put  and 
manory  requiratents,  avionics  system  interface  requirements,  and  flight 
test  aids  requiranents . 

o  Administer  to  external  interfaces  of  flight  critical  systems.  This 
includes  providing  a  support  function  in  terms  of  interface  definition, 
ejgjertise/consulting  on  interfaces  during  development  of  FCS  and 
providing  a  configuration  control  function. 

o  Provide  development  services  in  terms  of  requirements  interpretation 
during  design,  a  "fireman's"  role  during  detailed  design,  and  in  general 
provide  coherent  progressive  processes  for  system  development. 

o  Provide  expertise  in  systan  level  testing  by  assuring  that  the  testing 
level  is  established  at  the  level  the  requiranents  are  generated 
supporting  systan  or  subsystem  level  and  provide  a  systan  level  test 
leadership  role. 

The  role  of  the  flight  critical  systems  engineering  function  then  is  to 
provide  expertise  and  control  functions  in  the  requiranents,  design, 
implementation,  and  test  of  all  FCS  on  the  aircraft. 
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3.1.12  Flight  Test  Phase 


The  task  of  the  FligSit  Test  Phase  is  to  carry  out  all  testing  required  to 
certify  safety  and  perfomanoe  characteristics  in  the  actual  flight  environment. 

3.2  VERIFICATICg^  AND  VALIDATICM  OF  PCS  SCFTOARE 

3.2.1  Overview 

The  purpose  of  verification  and  validation  is  to  provide  systematic  assurance 
that  PCS  conputer  programs  will  perform  their  mission  requirements  efficiently 
and  correctly.  The  V&V  effort  serves  cis  a  program  acceptance  tool  in  providing 
higher  confidence  in  software  reliability,  compliance  between  specifications  and 
code,  and  adherence  to  accepted  standards.  When  performed  by  an  independent 
party,  independent  V&V  (IV&V)  provides  the  customer  better  visibility  into  the 
development  effort,  a  second  source  of  technical  ejpertise,  better  document 
quality  and  reduced  frequency  of  operational  change. 

Proper  performance  of  V&V  on  PCS  software  requires  an  understanding  of 
software  V&V  tools  and  techniques.  It  also  requires  a  strong  theoretical 
background,  and  experience  in  FCS  design  and  analysis.  This  background  and 
experience  are  needed  for  use  of  proper  analysis  techniques  and  putting  enough 
Qtphasis  on  critical  functions.  There  are  unique  problens  associated  with  the 
design  and  deA/elcpment  of  dicital  flight  critical  systans.  For  exanple,  in  some 
cases,  direct  discrete  design  is  required  as  opposed  to  discretization  of  an 
analog  design.  However,  it  is  also  necessary  to  account  for  pure  time  delays 
introduced  by  the  digital  design.  In  a  Digital  Flight  Control  System  (DFLCS) , 
the  validity  of  the  design  should  be  verified  throughout  the  operating  envelope. 
Sinple  frequency  response  measuranents  are  not  generally  adequate  for  this 
purpose,  particularly  where  variable  gain  scheduling  is  utilized  (and  if 
task-tailored  control  laws  or  reconfigurable  control  laws  are  to  be  irrplementcd)  . 

The  V&V  process  is  designed  to  address  each  critical  phase  of  the  software 
development  process.  Software  development  is  conprised  of  many  subactivities  oi¬ 
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tasks  and  the  V&V  proc3ess  assures  that  each  developnient  task  has  been  ccxipletely 
and  correctly  performed.  A  conprehensive  V&V  effort  to  assure  software 
reliability  will  include  the  following  basic  tasks: 

1.  Requirements  analysis:  assure  that  software  requiranents  nave  been 

correctly  derived  from  system  requiranents  and  that  the 

hardware/software  interface  requirements  are  ccrpatible. 

2.  Design  analysis:  assure  that  the  proposed  design  is  feasible  and  that 

proposed  mathematical  equations  and  algorithms  will  satisfy  the  software 
requirements. 

3.  Code  analysis:  assure  that  developed  code  is  a  correct  iitplonentation 

of  the  software  design. 

4.  Testing:  assure  proper  operation  of  program  modules,  software 

interfaces,  and  system  performance. 

A  detailed  description  of  each  V&V  activity  is  presented  in  the  following 

paragraphs . 

3.2.2  EX::s  Software  Requirements  Analysis 

The  definition  of  software  requirements  is  one  of  the  most  critical  phases  of 
the  software  development  process.  Verification  of  software  requiranents  is 
performed  to  ensure  that  system  and  interface  requirements  (documented  in  the 
system  and  subsystem  specifications)  are  correctly  allocated  to  software 
requirements  (documented  in  the  Conputer  Program  Development  Spiecification) .  The 
criteria  enployed  in  this  evaluation  include  corpleteness,  correctness,  and 

testability. 

Several  techniques  have  been  successfully  applied  to  the  verification  of 
software  requirements.  These  techniques  include: 

o  Independent  derivation  of  software  requirements  fran  system/ subsystem 
requirements. 
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o 


Corparison  to  standard  reference  systems  or  similar  systems  previously 
developed. 

o  Functional  simulations  and  modeling  of  process  allocation. 

o  Timing  and  sizing  analysis/  and  the  establishment  of  budgets  for  flight 
critical  system  parameters. 

o  Development  of  a  requirements  chart  which  identifies  interrelation¬ 
ships  between  requirements. 

A  V&V  test  criteria  is  selected  for  use  in  confirming  proper  inplementation 
for  ea^di  valid  software  requirement.  The  results  of  this  analysis  are  presented 
in  a  Software  Requirements  Analysis  Report  and  any  problems  detected  are 
documented  and  acted  \jpon. 

3.2.3  FCS  Software  Design  Analysis 

After  system  and  subsystem  requirements  have  been  allocated  down  to  software, 
the  software  design  phase  can  begin.  This  is  the  process  of  translating  software 
requiranents  into  a  basic  software  design  and  then  a  detailed  software  design. 
It  is  inperative  to  verify  that  the  proposed  software  design  satisfies  all  the 
software  requirements. 

The  software  design  defines  both  the  executive  control  logic  and  algorithms 
to  perform  each  software  function.  A  balance  of  analysis  techniques  must  be 
selected  to  verify  both  of  these  elements  of  the  software  design.  The  following 
design  analysis  techniques  have  proven  effective  in  detecting  design  errors: 

o  Correlation  and  traceability  between  design  elenents  and  software 
requirements. 

o  Functional  simulation  to  assess  design  integrity  and  process  allocation. 

o  Independent  derivation  of  equations  and  algorithms. 

o  Comparison  with  standard  references  and  models. 

o  Comparison  with  methods  which  have  been  proven  in  operational  systems. 

o  Mathematical  and  logical  analysis. 
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Design  analysis  techniques  to  be  utilized  for  any  pairticular  function  are 
dependent  upon  the  nature  of  the  function  (such  as  signal  filtering,  gain 
scheduling,  device  interfacing) .  For  exanple,  logic  analysis  techniques  are 
appropriate  for  executive  control  functions  while  mathesnatical  methods  are  more 
suited  for  numerical  functions.  The  preposed  design  of  each  software  function  is 
verified  by  using  the  selected  method  to  determine  the  extent  to  which  it 
satisfies  the  corresponding  software  requiranents.  Control  logic  is  sindlarly 
verified  to  ensure  proper  interaction  between  software  functions. 

3.2.4  Code  Analysis 

Analysis  of  the  developed  program  code  is  performed  to  ensure  that  the  coded 
representation  of  the  software  design  corresponds  to  the  verified  design.  The 
goals  of  the  program  analysis  are  to  ensure  that  the  coding  is  correct,  tfiat 
development  standards  have  been  followed,  and  that  no  latent  errors  have  been 
introduced  into  the  software  by  the  coding  process.  The  following  program 
analysis  techniques  are  examples  of  those  employed  to  identify  coding  errors: 

o  Version  coitparisons 

o  Text  editing  and  syntax  analysis 
o  Standards  auditing 

o  Equation  reconstruction 

o  Data  structure  ancilysis 

o  Flowcharting  and  logic  reconstruction 

o  Manual  code  inspection 

o  Software  sneak  anailysis 

Software  tools,  vtfiich  are  programs  designed  to  assist  the  analyst,  are  employed 
to  automate  many  of  the  above  program  analysis  techniques.  Software  tools  can  be 
used  to  help  identify  actual  or  potential  errors  in  the  developed  code,  and 
reformat  and  consolidate  information.  They  present  a  reliable,  cost-effective 
means  to  greatly  reduce  the  manual  program  analysis  techniques. 
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To  irBxjjTiize  the  visibility  of  the  software  development,  program  analysis  is 
performed  in  parallel  with  the  code  development.  This  is  achieved  by  analyzing 
the  incremental  code  deliveries  and  modifications  introduced  in  the  updated 
program  versions.  This  method  has  proven  to  be  an  effective  technique  for 
identifying  major  coding  prdolens  and  for  correction  early  in  the  code 

development  process.  Any  discrepancies  between  the  final  code  and  the  verified 

software  design  are  documented  and  acted  ipon. 

3.2.5  Flicriit  Critical  Svstgns  Software  Test 

To  ccnplete  the  validation,  tests  are  performed  to  determine  corpliance  with 
software  and  system  requiranents.  A  ccnprehensive  test  plan  is  developed  prior 
to  testing  and  tests  are  planned  to  achieve  the  following  objectives: 

1.  Verify  that  individual  software  functions  satisfy  the  corresponding 
software  requirements. 

2.  Verify  that  the  software/software  and  hardware/software  interface 

functions  are  properly  inplemented. 

3.  Verify  that  the  operational  system  possess  the  required  system 

capabilities  and  satisfies  the  appropriate  performance  requiranents . 

Tests  are  planned  at  module,  interface,  and  systen  levels  for  both  ncminal 
and  extreme  conditions  within  the  required  performance  limits.  Test  procedures 
are  developed  to  provide  a  detciiled  specification  of  the  exact  steps  to  be  used 
in  performing  the  test.  The  results  of  the  test  planning  activity  are  presented 
in  a  Test  Plan/Procedures  document.  Conprehensive  planning  is  the  foundation 
upon  vhich  effective  testing  is  based. 

Testing  is  conducted  by  following  the  exact  procedures  specified  in  the  Test 
Plan/Procedures.  Software  tools  are  aiployed  during  the  tests  to  provide  a 
testing  environment,  an  acceptance  criteria,  and  analysis  aid.  Test  results  are 
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recorded,  and  any  ancrnalous  results  are  confirmed  by  analysis,  documented  and 
acted  i5X)n. 

Testing  of  FCS  softvrare  generally  occurs  at  five  major  levels  during  software 
integration  and  test  phases  of  a  system  developnent.  These  are: 

1.  Execution  on  the  host  conputer. 

(Module  Testing) 

2.  Execution  on  an  emulator  of  the  tairget  ccnputer. 

(Module  &  Interface  Testing) 

3.  Execution  on  the  target  conputer. 

(Module  &  Interface  Testing) 

4.  Integrated  systen  simulation  testing. 

(Interface  and  Systen  Testing) 

5.  Flight  testing. 

(Systm  Testing) 

These  five  levels  are  normally  used  in  most  PCS  software  develqpment;  the  last 
three  levels  are  always  required.  Execution  on  a  host  ccmouter  usually  refers  to 
a  development  type  environment,  v^ere  code  can  be  examined  in  a  static 
environment,  modules  can  be  tested,  and  structure  of  the  code  can  be  tested  for 
proper  interfaces  between  modules. 

When  the  target  conputer  is  not  available,  the  target  ccnputer  is  emulated  on 
a  host  conputer.  This  emulation  testing  uses  identical  instruction  sets,  word 
sizes,  etc.  yet  provides  a  host  conputer  user  friendly  environment  in  which  test 
drivers  and  analyzers  can  be  "wrapped  auround"  the  emulator. 

Execution  on  the  target  conputer  provides  the  actual  ccnputer  environment  for 
the  software  execution.  Individual  modules  are  tested  and  integrated  to  a 
ccnplete  subsystem/ systen  to  allow  end-to-end  testing.  Execution  on  a  target 
conputer  in  conjunction  with  sane  external  inputs  (real  and  modeled  hardware 
devices)  is  often  referred  to  as  "hot-bench"  testing.  One  of  the  primary 
functions  of  "hot-)Dench"  testing  is  to  check  external  interfaces  and  software 
functional  performance  to  realistic  real-time  inputs. 
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Integrated  svsten  sirnulation  testing  is  performed  to  check  the  FCS  software 
in  the  full  range  of  operational  conditions.  In  this  phase  of  testing,  prototype 
or  actual  flight  coiputers  are  brouc^it  into  a  hot-bench,  iron-bird,  and/or  test 
rig  environment.  The  environment  is  made  to  present  the  operational  environment 
as  close  as  is  practical.  This  environment  includes  high-fidelity  aerodynamics, 
serisors  (normally  simulated) ,  hydraulics  and  actuators,  cockpit,  instrumentation, 
and  outside  view  presentations  for  pilot-cueing.  The  closed-loop  tests  that  are 
run  are  generally  pilot-in-the-loop  operations  to  verify  performance,  flying 
qualities,  and  to  confirm  proper  functional  dynamics  and  mode  sequencing.  In 
addition  to  these  tests,  "pilot  confidence”  testing  is  performed  vrtiere  the  pilots 
fly  realistic  missions  and  push  the  simulated  aircraft  to  its  safety  limits. 

Flight  testing  is  directed  toward  confirmation  of  performance  requiranents 
and  demonstrating  flight  safety.  Considerable  instrumentation  is  needed  to 
collect  data  which  can  be  analyzed  and  correlated  with  analytical  and  simulator 
predictions, 

3.2.6  FCS  Software  V&V  Tools  and  Techniques 

Table  3. 2. 6-1  broadly  summarizes  PCS  software  V&V  tasks,  applicable  V&V  tools 
and  techniques,  and  the  V&V  objectives.  An  extremely  large  number  of  tools  have 
been  and  continue  to  be  developed  to  aid  in  the  verification  and  validation  of 
software.  A  variety  of  these  tools  (static  and  dynamic)  are  listed  in  Table 
3, 2, 6-2.  Static  tools  exar.iine  some  aspect  of  specifications,  designs,  or  code 
without  executing  the  code.  These  tools  are  grouped  into  a  list  of  those  which 
examine  a  specific  property  and  those  which  examine  more  general  and  extensive 
properties,  A  dynamic  tool  performs  some  function  to  aid  in  testing  the  software 
when  the  program  is  actually  executed.  A  timing  analyzer  that  monitors  and 
records  execution  times  for  functions  is  an  exanple  of  a  dynamic  tool. 

Techniques  are  "standards  and  procedures"  used  in  development,  test,  and 
maintenance  of  software.  Table  3.2. 6-3  presents  some  standard  techniques  used. 
Development  and  maintenance  techniques  are  included  since  substantial  software 
reliability  can  be  obtained  by  attention  to  systematic  developmejit.  and 
documentat ion. 
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TABLE  3.2.6-1  V  &  V  REQUIREMENTS 


V4V  TASKS 

OBJECTIVE/PURPOSE 

APPLICABLE  TECHNIQUES/TOOLS 

System  Specification 

Evaluated  to  ensure  that  system/ 

o  Peqairenents  Analysis 

Verifications 

subsystem  considered  will 
fulfill  mission  goals  and 
objectives. 

o  Documentation  Review 

Control  Law  Analysis 

Assure  control  algorithms 

o  Requirements  Analysis 

adequacy.  Verify  equation 

o  Control  Law  Analysis 

accuracy;  evaluate  functional 
relationshif>s  and  functional 
performance  (timing,  sequencing, 
etc.) 

o  EiTailatlon/SljTulation 

Evaluation  of 

Evaluate  for  satisfactory 

o  Review  Management  Plan 

Development  Planning 

standards  4  practices, 
schedules,  planning,  controls, 
reviews,  audits,  01  change 
control,  problem  resolution,  V4V 

o  CPDP  Review 

Evaluation  of 

Preventive  Measure.  Sound 

o  Dooxnent  Review 

Softvrare  Development 

design,  coding,  and  test 

-  Standards 

Methodology 

techniques  reduce  nuttber  of 
errors  made  during  developrent. 

-  Plans 

-  Configuration  Management 
Provisions 

Software  Pequlreroents 

Requirements  evaluated  for 

o  Requirements  Analysis 

Verification 

adequacy,  oonpleteness, 
accuracy,  testability,  and 

o  Critical  Requirements 
Identification 

traoeablilty  to  higher  level 
^seclf Icatlons . 

o  Documentation  Review 

Software  Design 

Evaluate  development  products 

o  Design  Analysis 

to  ensure  technical  viability 

o  Performance  Analysis 

and  contribute  to  refinement 

o  Document  Review 

process.  Ehsure  software  design 

o  Top  Dexjn  Programning 

represents  a  clear,  consistent 

0  Sy^em  Level  Bailation 

and  accurate  translation  of 

0  Consistency  Chedeer 

software  requirements. 

o  Standardlxation 

code  Correctness 

Test  and  evaluate  developers 

o  Code  Analysis 

code  lisirq  independent  tools. 

o  Conparator 

Code  is  checked  for  errors, 

0  Corpiler 

onlssions  and  incorrect  trans- 

o  Interface  Checker 

latlons.  Evaluate  logic,  file 

o  Document  Review 

structuring,  execution  paths  aixJ 

o  Cross  Refererxe 

limitations,  interfaces,  etc. 

0  Cross  Assenbler 

htechlne  level  open-loop  tests 

0  SliTulation 

and  unit  and  nodule;  closed-loop 
at  s^ieyst  era/system.  Examine 
timing. 

o  Instruction  Trace 

Identify  unexpected  paths  for 
information  flow  thixxjgh  a  pro¬ 
gram  by  analyzing  the  clues 
characteristic  of  6nea)c  paths  in 
retworlc  trees/flow  graphs. 

o  Sneak  Analysis 

Software  Validation 

lietcrTTvine  whether  all  software 

o  Test  Plan/Prooedure  Review 

and  system  perfonnanoe#  inter- 

o  Test  Case  Generation 

0  Development  Tests 

face,  functional  and  test 

o  Hot -Bench  Sixaiiator 

0  System  Tests 

requirements  are  fulfilled. 

o  Kiinframe  Sirrulation 

0  Flight  Tests 

o  Iron-Bird  Siiriilat  inn 

o  Control  Lane 

Response 

o  Handling  Quantities 

0  Functional  Tests 

-  ENncry  roqoiremcnt  is 
acfcxjuatcly  tested 

-  All  subsystems  are  properly 
integrated 

-  All  6>-stcro  responses  are 
adfxjunte  for  performanne  and 
safety. 

o  Aircraft  Fliqiit 

o  B£?<jlindancy 

Insure,  through  independent 

o  S>~trm  ly-'w!  UnUatit'n 

KuiagcrncT>t 

te.'Tt, inn,  tli.it  nM/FTi  system  rm-^.s 

o  Iin:;  I  '.'rtil.-'t  io.;'’. 

Failure  Kinagr^nnnL 

desKji  rrvjui  n'^n'iU  r-  f<.»r  wd!  st 

liisi/iint  i« 'I..;  ill 

jif *i  lonn  jwsT anit':  r  ie  «^i.ilysis  ot 
.*.1  . 

FnT 

TABI£  3.2. 6-2  VERIFICATICW  AND  \M.IDATia^  TOOLS 


general  static  tools 

DYNAMIC  TOQlS 

ClRCULAp  PFFpRENCe  CHECKER 

ACCURACY  analyzer 

data  F  l  0  W  pat  wing 

cor:E  cok/rarator 

OOCUMENTATiCN  ANO 

EMULATION 

CROSS-Rtf-ERENCE  CHECKER 

construction  systems 

SIMULATIONS 

DATA  BASE  analyzer 

E  Di  TOR 

•  CCUPD  TE  u 

( LOW  Char  1 6  R 

EORMAL  LANGUAGES  with 

•  hybrid 

iNTERfACL  CHECKER 

syntax  analyzers 

•  TEST  BED  (IRON  e.RD) 

•MODULE  INVOCAT  lON 

•  RECuiREMENTS 

TEST  Data  GENERATOR 

program  flow  analvzeR 

•  S'^EC'EiCaTiONS 

TEST  DRIVER 

SET/USE  CHECKER 

•  program  DESIGN 

TEST  EXECUTION  MCN'TOR 

UNITS  CONSISTENCY  CHECKER 

•  program  code 

TEST  RECORD  GENERATOR 

unreachable  code  vector 

sneak-path  analyzer 

TIMING  analyze  R 

SYMBOLIC  evaluator 

Theorem  prover^ 

TOOL  DEFINITK^S 

Accuracy  analyzer  -  analyzes  numerical  calculations  for  req'd  accuracy 
Circular  reference  checker  -  modules  calling  each  other 
Code  cotparator  -  differencing  between  versions 

Cross-reference  checker  -  calling  of  modules;  external  variables  called 
Data  bases  analyzer  -  module  accesses  to  data  bases;  unused  elements 
Data  Flow  Pathing  -  trace  execution  sequence  for  variable (s)  in  flow 
Documentation  &  constructions  -  Auto  documentation;  Consistent  data  pool 
Eklitor  -  Analyze/extract  information/relationships  from  source  programs 
Elnulations  -  Syst«n  level  model  generated  from  requirements,  not  design 
Flow  charter  -  show  logical  construction  of  program 
Formal  Languages  -  Program  structures  and  rules 
Interface  Checker  -  Check  Range,  limits,  scaling  of  variables 
Module  Invocation  Tree  -  Establishes  call  hierarchy  with  system 
Program  flow  analyzer  -  statistics  on  usage;  estimate  execution  tine 
Set /Use  checker  -  Checks  for  variables:  set,  not  used;  &  used  before  set 
Simulations  -  Test  characteristics,  algorithms,  functions,  performano:^ 
Sneak -Path  Analyzer  -  Looks  for  unexpected  paths 

Symbolic  evaluator  -  reconstructs  equations  relating  output  to  input 
Test  data  generator  -  produces  test  cases  to  exercise  the  system 
Test  driver  -  controls  the  execution  of  a  program 

Test  execution  monitor  -  collects  data  and  cottpares  to  exf-ected  results 
Test  record  generator  -  analyzes,  reduces,  and  formats  results 
Tlieorem  provet  -  axioms  used  prove  assertions  stattxl  for  a  path 
Timing  analyzer  -  monitors/records  run  tinK'  of  functions  and  routines 
Units  consistency  checker  -  variable  exf^ressions  (units)  checked 
Unrriaclrible  c:ode  detecdior  -  looks  for  code  which  cannot  lx*  cxecutotl 


TABLE  3.2. 6-3 

DEVELCPl^Elir  AND  V&V  TECHNIQUES 


Abstractions  and  hierarchies  to  reduce  complexity:  abstractions  such  as 
trees  are  used  to  make  the  design  sinple  and  clearly  defined 
Checkout  (debug)  testing:  function/module  testing  before  integration 
Constructive  design  approaches:  (eg.  formal  design  language) 

Critical  Design  Review:  oral  demonstration  of  detailed  design 

Data  flow  diagram,  structure  chart:  shows  flow  of  data  in  program  and 
hierarchial  organization 
Descriptions  or  documentation 

Design  guidelines,  test  guidelines,  &  coding  guidelines 
Design  standards,  coding  standards 

Functional  capabilities  list:  module  description  of  functions  to  perform 
Integration  testing:  code  test  after  modules  are  assentoled;  I/O  structure 
Organization  as  finite  automata:  provides  clear  structure  of  functions  for 

Fix:s 

Qualification  audit 
Singularities  and  extremes  testing 

Syntoolic  execution:  performed  on  special  functions  such  as  mode  logic 
Systems  concept  review:  orail  demo  of  initial  concepts,  trade-offs,  etc. 
Validation  testing:  fined  demo  in  sinaiLation  environment 
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Evolvement  of  Conputer  Aided  Software  Engineering  (CASE)  tools  continues 
towards  providing  software  developniKit  with  the  environment  of  an  integrated 
tool  set  which  includes  planning/  analysis,  design,  documentation,  static 
analysis,  prototyping,  cinnamic  ancilysis,  simulation,  and  construction  of 
executable  systans.  (See  Referenmoe  2)  Table  3. 2. 6-4  presents  sane  of  the 
^plicable  CASE  tools  that  are  caimercicilly  available.  These  tools  span  the 
gamut  from  powerful  linear  systems  anailysis,  prototyping  and  code  generation  to 
those  vAiich  provide  ciids  in  the  form  of  a  data  dictionary,  creating  data-flow 
diagrams,  process  specifications,  and  grafAiic  documentation  of  design. 
Evolving  CASE  tools  are  providing  ways  to  help  manage  the  cottplexity  of 
large-scale  software  systems.  The  tools,  like  the  methods  they  iraplanent,  are 
not  final  solutions  but  are  aids  to  providing  a  more  friendly  software 
development  and  test  environment. 


Table  3.2. 6-4  REPRESENTATIVE  CASE  TOOLS,  METHCOOLOGIES,  AND  LIFE  CYCLES 


CASE  TOOL 

METHODOLOGY 

LIFE  CYCLE 

i 

TEAMWORK  OS/2. 3.0 

DeMARCO,  WARD/ 

SCHLAIR,  CONSTANTINE 

PLANNING,  ANALYSIS 

DESIGN 

CXELERATOR  1.84 

YCURDON,  GENE/SARSON 

PLANNING,  ANALYSIS 

DESIGN 

POSE  4.0 

YOURDON.  GANE/SARSON, 
CONSTANTINE,  FINKELSTEIN, 
INFORMATION  ENGINEERING 

PLANNING  ANALYSIS 

DESIGN.  CONSTRUCTION 

TRACEBUiLDER  II 

TRACES  SOFTWARE 

REO’S  FORWARD  8 

BACKGROUND 

ALL  PHASES  OE 

SOFTWARE  DEVEl_OPMEN’ 

in  I  Ai  D  r^rTi  [  ms, 

irjc 

1  INEAR  MODI  l  ING. 

SV.Sri  M  GUILD,  AU(v)-(,C()! 

GENE  RAT  ION 

CONCi  E  ;  i:;.i  ui  ■  .m-, 

:  '  Tn  t  A  n  *'  ' 

vf 

Besides  attenpting  to  eaise  the  software  programmer' s  burden,  code 
reusability  is  another  issue  which  CASE  develcpers  are  targeting.  Software  now 
constitutes  90%  of  an  electronic  systems  functionality  (vs.  10%  during  the 
1960s) .  With  so  much  code  being  written,  CASE  tool  manufacturers  are 
developing  tools  to  tackle  reusability  problems.  These  tools  are  part  of  a 
Icirger  system  of  front-end  tools  to  streamline  the  programming  process  and  make 
it  more  efficient. 

Fundamentcilly,  CASE  tools  must  meet  several  criteria  in  order  to  be 
successfully  adc^ed  as  part  of  a  software  developer's  tool  kit.  These 
criteria  include: 

o  Break  down  complexity  of  requirements  and  designs  into  manageable 

ccsiponents. 

o  Presentable  to  several  audiences  inclixlLng  end-users  and  contracting 
organizations. 

o  Cheaper  than  bn T ding  the  real  thing  as  ccnpared  to  conventional 

software  development  approach. 

o  Quantitative  and  Verifiable  with  respect  to  requironents  traceability 
and  performance  criteria. 

o  Graphically  oriented  to  provide  more  easily  understood  graphical 
illustrations  of  design. 
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3.3  TASK  1  RESULTS:  PEQUIREWENTS  Em  ET.IGHT  CRITICMi  SYSTEMS  SOFTWARE  V&V 

3.3.1  General  Reouireftents  for  Fliodit  Control  Svstetns 

General  requirements  for  fli^t  critical  systems  are  generally  broken  into 
two  parts:  mission  requirements  and  safety  requirements.  For  exanple,  fli^t 
control/engine  control  is  nornally  first  driven  by  safety  requirements  and 
secondly  by  mission  requiranents .  For  most  other  systems,  mission  requirements 
are  given  more  design  attention  relying  on  fli^t  control  (and  the  pilot)  to 
provide  the  needed  margins  of  safety.  Commonly  used  specifications  for 
probability  of  loss  of  an  aircraft  due  to  a  fli^t  control  systm  failure  is: 

Floss  <  1  ^  10  per  fli^t  hour  (milztary  aircraft) 

Floss  <  1  X  10”^  per  fli^t  hour  (civil  aircraft) . 

With  the  advent  of  digital  fli^t  control  systans,  these  failure  probabilities 
have  been  applied  to  the  total  digital  systan  including  the  software.  To 
achieve  these  numbers  for  the  entire  flight  control  systan  safety,  the  prob¬ 
ability  of  aircraft  loss  due  to  software  failure  must  be  even  lower  to  achieve 
this  hi^  safety  itargin.  Elnphasis  in  past  and  current  developments  has  been 
placed  on  fault  avoidance  and  fault  tolerance  techniques  in  software  design. 

With  the  advent  of  the  highly  coipled  flight  critical  systems,  this 
distinction  between  flight  control  and  other  flight  critical  systans  regarding 
safety  consideration  is  diminishing  and  fault  avoidance/tolerance  issues  must 
be  applied  in  all  flight  critical  systems. 

3.3.2  FCS  Design  Trends  Impact  On  Software  V&V  Recaairements 

Design  techniques  vhich  cu:e  currently  used  to  address  safety  aspects  of  PCS 
software  include  fault  avoidance  and  fault  tolerance.  Fault  Avoidance 
techniques  apply  structured  design  methods  that  incorporate  rigorous 
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quality  control  and  systematic  testing  of  the  software  to  insure  that  the 
probability  of  a  software  "bug"  being  introduced  or  ranaining  undetected  during 
the  software  design  and  development  process  is  extremely  low.  One  technique  of 
fault  avoidance  is  the  use  of  very  small  and  very  simple  modules  vAiich  are 
relatively  easy  to  veri^.  It  is  cotmonly  assumed  that  high  integrity  can  be 
achieved  throu^  use  of  such  techniques.  In  practice,  however,  no  matter  how 
carefully  the  softWcu?e  is  ctesigned,  it  is  ixtpossible  to  establish  that  it  is 
ccnpletely  error  free  because:  (1)  the  larger  number  of  possible  states 
preclude  exhaustive  testing;  and  (2)  the  usual  statistical  analysis  methods 
which  are  useful  in  hardware  development  are  not  applicable  to  software 
development.  Therefore,  Fault  Tolerance  is  introduced  into  design  to  cope  with 
faults  which  are  not  discovered  during  design  and  inplementation  process.  A 
good  fault  tolerant  design  should  prevent  any  remaining  faults  frcm  having  a 
catastrophic  effect  on  the  systen. 

Flicfit  control  systems  requirements  have  and  will  continue  to  drive  safety 
aspects  in  software  design.  While  requiuOTents  for  digital  flight  control  are 
not  unique,  collectively,  they  represent  the  most  dananding  requirarents  in 
guidance  and  control  applications.  Real-time  closed-loop  operations, 
multi-mode  design,  bandwidth  variations,  multi-loop  design,  and  tight 
interface/reliance  on  sensor  systems  are  seme  of  the  characteristics  of  flight 
controls  vrtiich  drive  overall  software  design  conplexity.  These  factors  along 
with  the  trends  in  cdrcraft  systems  and  avionics  systems  design  toward  the 
increased  use  of  relaxed  static  stability  and  integrated  avionics/control 
functions,  place  even  more  inportance  on  software  fault  avoidance  and  fault 
tolerance . 

The  near  term  trends  in  fli^t  control  systems  (FLCS)  are  expanding  the  use 
of  real-time,  on-board  optimization  and  intelligent  controls  to  achieve  high 
performance  and  provide  for  damage  tolerance  and  self-healing  designs.  These 
near  term  FLCS  already  are  addressing  the  inner-loop,  outer-loop,  and 
redundancy  management  functions  shown  in  Table  3.3-1. 
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TARTF.  3.3-1  NEAR  TEFiM  TRENDS  IN  FLCS  FUNCTIOIS 


1  FUNCTIONS 

INNER-LOOP 

OUTER-LOOP 

REDUNUANv^^y  MANAGEMENT 

Z 

•  RELAXED  STATIC  STABILITY 

•  TF/TA/OA 

.  ANALYTIC  REPUNDANCY 

.  GUST  LOAD  ALLEVIATION 

•  Al  BASED  DECISION  MAKING 

.  Ml  BASED  TECHNIQUES 

.  RIDE  QUALITY 

•  INTEGRATED  CONTROL 

•  FLUTTER  MODE  CONTROL 

•  OPTIMAL  FLIGHT  PATH  CONTROL 

•  INTEGRATED  CONTROL 

FCS  Integration  has  an  even  more  challenging  inpact  on  software.  Table 
3.3-2  surmarizes  seme  of  the  areas  related  to  integration  and  the  related 
iiipacts  of  fault  tolerance. 


Table  3.3-2  EXAMPLES  CF  IMPACTS  OF  INTEGRATION  OFfCEPTS  (F  CCMPIEXITY 


CONCEPT 

DESCRIPTION 

CRUCIAL  FUNCTION 

IMPACT  1 

DISPERSED. INTEGRA!  ED 

FLIGHT/NAVIGATION  SENSORS 

SHARING  OF  STRAPDOWN 

NAVIGATION  SENSORS  WITH 

FLIGHT  CONTROL 

MORE  COMPLEX  TCS 

ALGORITHMS  FOR  SENSOR 
NORMALIZATION.  REDUNDANCY 
MANAGEMENT.  &  SURVIVABLE 
DISPE  RSION 

INTEGRATED  FLIGHT/ 

SHARING  OF  INFORMATION 

FLT  CRUCIAL  ENGINE  CTRL 

PROPULSION  CONTROL 

BETWEEN  ENGINE  AND  FLIGHT 

CONTROL  SYSTEMS.  VECTORED 

j  1 

THRUST 

REAL  TIME  OPTIMAL  CONTROL 

LOW  ALTITUDE  AUTO  FLIGHT 

I  FLIGHT  PATH  MANAGEMENT  i 

FOR  TF/TA. COMPLEX  ROUTF 

MANAGEMENT  IS  CRUCIAL. 

DECISIONS  AT  LOW  ALTITUDE 

SENSOR  BLEND  CGNCEPTS- 
FLT  CRUCIAL.  HUGE  TERRAIN 
DATA  BASES  (eg  DTMl  ARE 

FLIGHT  CRUCIAL 

1 

VEHICLE  MANAGEMENT  1 

0  (  F  P  C 

o  complexity 

!  SYSTEMS  ' 

o  UTILITIES  SYSTEMS  MGMT 

o  HIGHLY  IN  T  L  HACl  1  VL 

o  INTE3  CONTHOl  FUNCIIONS 

o  HLCONMGURATION 

o  INTEG  MAIN! /OIAGNOGT  ICS 
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3.3.3  Technology  Impacts  on  V&V  Recnjiranents 

Technology  advancements  in  flight  critical  systans  software  design  and  in 
software  verification  and  validation  testing  have  been  significant  over  the 
past  15-20  years.  The  ctevelopitienu  and  use  of  digital  systans  in  flight 
critical  systans  applications  have  pushed  verification  and  validation 
techniques  to  meet  the  danands  of  testing  increasingly  ccnplicated  systems.  A 
number  of  accepted  validation  testing  methods  are  used,  but  verification  and 
validation  technology  generally  lags  the  advances  being  made  in  the  development 
of  PCS  software.  Currently,  convenient  verification  and  validation  methods  and 
tools  are  lacking  for  multi-channel  and  highly  integrated  systens. 

Trends  and  projections  in  flicht  control  systen  design  and  inpacts  on 
validation  have  been  presented  in  Reference  3  and  are  summarized  in  Table 
3.3-3.  The  trends  presented  in  the  table  are  very  realistic  and  provide  moti¬ 
vation  for  developing  inproved  FLCS  verification/validation  techniques 
simultaneously  with  evolving  fli^t  critical  systems  concepts.  It  has  long 
been  advocated  that  many  of  the  corrplications  associated  with  V&V  of  PCS 
software  can  be  avoided  by  anticipating  the  V&V  requirements  early  in  the 
design  process  and  by  using  many  of  the  evolving  structured  V&V  techniques  and 
tools  discussed  in  Section  3.5. 

3.4  TASK  2  RESULTS:  DATA  COLI£CTICN 

The  primary  efforts  involved  in  the  data  collection  task  were  to  perform 
literature  reviews  on  PCS  software  deA/elopment  and  V&V  methods  and  to  conduct 
numerous  focused  interviews  with  PCS  developers  and  key  government  PCS 
experts.  In  order  to  help  organize  and  focus  these  efforts,  PTI  developed 
checklists  (see  i^^pendix)  addressing  the  classes  of  information  to  be 
gathered.  The  checklists  addressed  specifics  on  current  developmental 
approaches  used  for  PCS  software,  available  software  analysis  tools  and 
techniques  being  used,  facilities  eind  support  requirements,  problems  most  often 
encountered  during  development,  experiences  in  the  development  of  PCS  software, 
and  perceptions  of  what  PCS  software  development  proc-rams  would  require  because 
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Table  3.3-3  Forecasts/Projections  for  FCS  in  21st  Century  Indicate 

(source  AGARD  WG09) 

Significant  increases  in  caiputer  power  will  cause  itiajor  ejqsansion  in 
scope  and  chciracter  of  onboard  systems 

Development  of  circhitectural  branches  within  redundant  systens  will 
add  verification  and  validation  complexity 

Redundancy  in  management  functions  (e.g.,  voting  planes,  etc.) 
Qtibedded  in  special  purpose  HW  isolated  from  FLCS  will  change 
verification  and  validation  conplexity 

Highly  fault-toleraint  HW  designs  that  provide  "dynamic  redundancy" 
changes  the  scope  and  conplexity  of  verification  and  validation 
efforts 

Elnbedded  r^licated  or  dissimilar  subchannels  for  self  monitoring 
could  reduce  redundancy  management  complexities  at  higher  rates 
Increased  throu^put  and  emerging  new  architectures  are  allowing 
sensor  fusion  with  information  integration  and  display,  requiring 
expanded  EX:S  verification  and  validation  roles 

Trends  are  towards  systems  highly  integrated  through  FLCS  because  of 

mission  and  perfonricinoe  benefits  —  leads  to  more  testing  at  system 

levels,  interdisciplinary  expertise,  and  pilot  involvement 

Increase  of  control  effectors  and  reduction  in  actuator  redundancy 

levels  for  self  repair/reconfigurable  flight  control 

High  bandwidth  PCS  for  active  vibration  and  load  control  have 

associated  characteristics  vrfiidh  inpact  other  FCS 

Hypersonic  vehicles  require  VTC  to  have  total  vehicle  enerq^’/thermal/ 
trajectory  managonent  integrated  with  FLCS 

Decision-Aiding  systans  in  a  real-time  environment  require  validation 
of  knowledge  base  vhich  currently  has  no  accepted  validation  methods 
Interfaces  and  internetting  to  unmanned  vehicles  leads  to  additional 
corrplexities  and  veriiication  and  validation  requirenents 
Boundaries  between  non  flight  critical  and  flight  critical  systems  are 
projected  to  dissolve  with  increasing  integration  of  systems 
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of  the  latest  technology  being  incorporated  into  FCS.  The  checklists  also 
included  a  list  of  applicable  Government  standards  vdrLch  address  FCS  software 
development.  A  catalog  of  ^plicable  software  tools  used  in  development, 
verification  and  validation  of  fli^t  critical  systens  software  was  prepared 
and  included  in  the  checklists  to  aid  with  tool  identification. 

Literature  searches  v^re  performed  including  review  of  publications  and 
recent  articles  at  the  AFTECH  Library,  review  of  software  development  and 
fli^t  critical  systems  technical  journals,  and  a  Defense  Technical  Information 
Ctenter  search  on  related  subjects.  Hundreds  of  related  article  abstracts  were 
scanned  for  appropriate  subject  material.  In  order  to  provide  an  orderly  way 
of  tracking  and  retaining  pertinent  information  gained  from  these  reviews  a 
Data  Base  Managonent  System  (DBMS)  was  inplemented.  This  DBMS  was  used  then  to 
help  organize  and  record  the  many  identified  sources  of  data,  tools  and 
techniques.  This  DBMS  was  created  on  DBASE  1 11+  and  hosted  on  an  IBM 
ccnpatible  PC. 

A  number  of  the  reports  and  documents  specifically  directed  at  flight 
critical  systems  development  and  its  verification  and  validation  were 
identified.  Two  very  recent  AGARD  reports,  "Language  Sipport  Environments  For 
Guidance  And  Control  Systems"  -  Final  Report  Working  Group  08,  and  'Validation 
Of  Flight  Critical  Control  Systems"  -  Report  of  GCP  Working  Group  09,  were  very 
helpful  and  current  on  many  of  the  flight  critical  systsns  requirements,  flight 
critical  systens  trends,  development  approaches,  and  verification  &  validation 
practices.  The  "Handbook  -  Volume  I  Validation  of  Digital  Systems  in  Avionics 
and  Flight  Control  I^lications"  and  the  "Digital  Systens  Validation  Handbook 
Volume  II",  (References  4  and  5,  respectively)  both  published  by  the  US  DOT  are 
also  excellent  sources  of  material  for  verification  and  validation  testing 
practices  used  in  modem  flight  critical  systems.  Another  informative 
reference  that  serves  as  a  good  primer  on  flight  control  software  validation  is 
the  "Digital  Flight  Control  Software  Validation  Study",  (Reference  6)  an  AFFDL 
technical  report.  Discussions  of  recent  Coirputer  Aided  Software  Engineering 
tools  were  presented  in  several  articles;  the  book  by  A. S. Fisher  titled  "CASE" 
gave  an  excellent  sunmary  of  vhere  CASE  tools  are  now  and  their  current  trends. 
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In  depth  interviews  were  cx)nducted  with  key  goveminent  and  industry  experts 
in  the  developnent  of  flight  critical  systems-  These  interviews  included 
personnel  at  Wri^t  Research  Development  Center  Flight  Dynamics  Laboratory,  Air 
Force  Flight  Test  Center,  NASA  Dryden  Flight  Research  Facility,  McAir,  General 
Dynamics,  Rockwell  International,  Honeywell,  Softech,  and  High  Plains.  The 
prepared  checklist  was  used  to  guide  the  discussions  on  FCS  software 
verification  and  validation  tools  and  techniques. 

Cotments  gathered  from  the  above  sources  have  been  summarized  below  in 
terms  of  V&V  Drivers,  Development  and  V&V  Pfethodologies,  Higher  Orcter 
Languages,  Development  and  V&V  Tools,  Validation  Testing,  Future  FCS  Softwaire 
Considerations,  and  Problens/Lessons  Learned. 

FCS  SOFTOARE  V&V  DRIVERS 

o  FCS  move  to  digital  inplenentation  increases  ccmplexity  of  V&V  effort, 
o  Increased  systens  integration  acccnplished  through  software, 
o  Move  to  Ada  requires  tpdate  of  V&V  tools  and  techniques, 
o  Cotplexities  of  Failure  Managonent  /  Self  Healing  requires  carefu],  test 
planning  to  get  adequate  testing  coverage, 
o  Maintenance  of  PCS  software  requires  extensive  V&V  capabilities, 
o  Move  towards  transportability  (MIL-STD-1750  kills  this  area) . 
o  Integration  of  software  with  hardware  conplicates  V&V  testing. 

FCS  SOFTWARE  DEVELOPMENT 

o  2167A  waterfall  chart  represents  how  development /test  is  performed, 
o  Rapid  prototyping  is  useful  early  design  aid. 

o  Projects  organized  along  the  lines  of  how  the  developnent  proceeds 
are  desirable.  One  good  example  is:. 

-  Aerodynamic  Stability  and  Control 

-  Control  Law  Design  and  Analysis 

-  Flight  Critical  Systons  EIngineering 

-  Flight  Control  Mechanization  and  Software 

-  Flight  Control  Hairdware  Design 
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-  Flight  Control  Systems  Test 

-  Flight  Control  Operations 

o  Flight  Critical  Systens  Engineering  function  is  an  increasingly 
iiiportant  fijnction  to  administer  to  needs  of  flight  critical  portion 
of  aircraft.  It  provides  a  continuum  of  understanding  across  the 
development  organization. 

o  Quality  Functional  Deployment  is  good  formal  planning  and  documenting 
why  you  have  done  vdiat  you  have  done. 

o  Some  developers  feel  strongly  that  the  software  development  should  be 

kept  with  the  fli^t  control  engineers.  In  principal,  other  developers 
agree,  but  that  the  software  engineer  is  better  equipped  to  write 
software  —  therefore  training  software  engineers  in  the  development  of 
FCS  software  is  mandatory. 

USE  OF  HIGHER  ORDER  LANGUAGES  (HOLS) 

o  HOLs  are  in  general  good.  However,  once  code  is  reccrpiled,  it  is 

difficult  to  say  that  new  code  is  good  versus  an  assembly  language 
patches  approach. 

o  One  advantage  of  HOL  is  that  it  allows  the  system  analyst  (flight 

control  engineer)  to  read  or  even  develop  the  code.  This  avoids  the 

problon  of  misconmunication  between  the  designers  and  the  software 
inplementors, 

o  The  proper  place  to  standctrdize  is  the  language.  Ada  has  some  problons 
(Ada  tasking,  rendezvous,  etc.),  but  you  do  not  have  to  use  all  of  the 
capabilities  of  the  language. 

FCS  SOFTWARE  DEVELOPMENT  AND  V&V  TOOLS 

o  FCS  software  development  is  moving  towards  provision  of  control  law 
block  diagrams  to  the  FLCS  houses  for  automatic  code  generation.  GE's 
program  called  FASTER  directly  generates  1750A  assorbly  code, 
o  FLCS  Tools  xjsed; 

-  Ctrl-C 

-  Matrix  X 

-  GenAir:  Generic  Aircraft,  a  McAir  tool.  This  uses  general 
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control  laws  and  general  aircraft  configuration  performance.  Used 
for  mission  performance  evaluation. 

-  Modular  Design  &  Analysis  Tools 

-  Nonlinear  cdrcraft  model  simulation 

-  MATLAB,  EASY5,  lyMKEXx,  CTRL-C 

-  Use  of  script  files:  set  of  coninand  files  that  go  to  simulation 
ccnputers 

-  Recording  systen  for  all  parameters  in  the  simulation  system. 

o  Test  tools  that  lose  actual  fli^t  boxes  and  automate  testing  are 
evolving.  The  FUlly  Automated  Tester  and  Error  Reporting  (EATER)  is  an 
exaitple  vAiioh  compares  control  laws  in  Fortran  versus  assembly, 
o  TAE  (Transport  i^lication  Executive)  is  a  NASA  Goddard  Flight  Center 
tool  used  in  Simulated  Rapid-Prototyping  Facility  (SRF)  of  the  WRDC 
Flight  Dynamics  Laboratory.  It  has  standard  I/O  for  a  program  and  runs 
on  a  dozen  different  computers, 
o  FCS  Integration  Tools/l^thods  Inclixie: 

-  Basic  documentation  tools. 

-  A  lot  of  simulation  for  R&M  testing. 

-  In-house  fault  tree  analysis. 

-  1553  analysis  tools. 

-  Flow  diagrams  &  analyzing  timing  between  functions. 

FCS  VALIDATION  TESTING 
o  Utilizes  a  bottom-ip>  philosophy. 

o  Starts  with  lowest  level  code  and  progresses  to  syston- level  testing, 

o  Provides  verification  at  one  level  before  progressing  to  the  next, 

o  Test  plans/prooedures  eire  prepared  for  each  phase/level  of  testing, 
o  Test  results  documented,  discrepancies  documented,  investigated  and 
acted  upon. 

o  approach  ensures  systan  operates  as  designed  and  is  flight  worthy. 
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o  Provides  total  visibility  of  systan  developnnent  whicb  allows  better 
managanent  control. 

o  Handling  qualities  quantitative  solutions  have  failed, 
o  Integrated  system  V&V  evaluates  system  functional  requiranents : 

-  closed  loop  test  environrrent  simulates  the  dynamic 
behavior  of  the  air  vehicle 

-  actual  fli^t  hardware  is  used  vhere  practical 

-  verifies  closed  loop  cfynamic  response 

-  used  for  handling  qualities  evaluation 

-  evaluate  pilot  vehicle  interface  evaluation 

-  used  for  failure  managanent  evaluation 

-  evciluates  fadlure  modes  and  effects  tests 

o  The  ability  to  test  back-to-back  software  (ie.  previous  CFP  vs  updated 
OFP)  offers  many  benefits  during  validation. 

FtmJRE  FCS  SOFTWARE  OCMSIDEmTIONS 

o  Future  PCS  systems  will  most  likely  have  to  address  interfacing  with 
existing  systems.  A  wide  generation  of  FCS  corputers  exists.  Some 
older  ones  cannot  support  HOLs. 
o  Development  contractors  are  moving  towards  using. 

-  RISC  cortputers;  currently  there  is  not  adequate  support  tools 
in  this  environment. 

-  Ada  language  programming. 

o  Transportable  software  is  being  addressed. 

-  Software  ccnpilers  are  currently  a  problen  here. 

-  Timing  is  one  of  the  most  critical  elements  in  flight 
critical  software  and  this  effects  transpoihability. 

o  Vehicle  Management  Systems  (VMS)  is  the  new  focus  in  FCS 

-  Developers  must  be  realistic  about  what  they  propose  and  use. 

-  The  ccxnbinatorial  considerations  make  it  inpossible  to  test 
all  combinations 

o  Developers  are  looking  at  real  needs  of  cortmon  Module  approach. 

-  Designing  test  stations  that  will  test. 
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-  Using  language  translators  for  new  front -ends  to  test  tools. 

-  FLC  filters  have  adreacfy  been  transported. 

-  Ada  will  help  "Conmon  Module's"  in  the  future. 

o  Redundancy  &  Monitoring  (R&M) 

-  Test  coverage  is  the  problen  here. 

-  There  is  the  question  of  Quad  vs  Triplex.  Triplex  can  meet  the 
1  X  10“^  problen,  but  it  is  difficult  to  meet  a  requiranent  of 
fail-op,  fail-(p  without  going  to  a  Quad  system  — 

is  this  requirement  unduly  irtposed  on  flicfit  critical  systens? 

-  A  quad  voter  runs  twice  as  long  a  triplex  voter. 

-  Software  conplexities  at  least  double  for  every  channel  added. 

o  Total  System  Integration 

-  New  techniques  are  being  xised  for  robust  control  laws, 
multi-axis,  integrated  fli^t  propulsion  methods,  multi-thrust 
vectoring,  and  self  repaiiring. 

-  Need  a  manageable  way  of  dealing  with  reconfiguration  &  fault 
isolation,  reconfiguration  control,  and  advance  control  design 
software. 

PRmLEMS  AND  LESSONS  LEARNED 

o  Problens  arise  in  specifications  across  flight  critical  systems 

interface. 

o  Use  of  simulation  for  testing  integrated  systons  is  questionable. 

-  Can  not  simulate  EO  and  radar  devices  that  well.  Models  can 
be  built  for  it,  but  usucilly  they  are  single  thread. 

-  Sensors  &  integration  depend  on  models  for  high 
technology  sensors.  Modeling  is  very  difficult. 

o  Use  of  simulation  for  V&V 

-  Organizations:  people  and  equipment  have  to  be  plarined  and  adequate. 

-  There  is  always  a  reluctance  to  change  a  simulator  once  things 
are  up  and  running.  Some  flexibility  is  required. 
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People  who  have  tested  systens  have  to  put  information  back  into  the 

loop.  The  problems  that  were  encountered  and  how  they  were  solved  is 

not  reported.  Only  the  good  peirt/results  seem  to  get  published. 

Result  of  flying  qualities  testing  has  produced  much  disinformation. 

One  very  large  need  is  requirenents  &  specifications  for  control  laws. 

-  There  is  a  lack  of  a  reasonable  MIL-Spec  for  flicfit  control. 

-  PIO  prediction  is  an  exanple  of  this. 

-  Mil  Prime  Standauxl  8785-C  is  not  adequate,  it  is  a  back-vp 
guide. 

Design  groqp  practices  could  be  inproved: 

-  Must  think  ahead  as  to  the  way  things  will  be  tested. 

-  Lack  of  documentation:  integrated  system  documentation 
defining  how  systems  work  together  is  needed. 

Most  errors  are  in  design.  These  are  generally  found  in  systems 

integration  testing. 

-  B-2  put  a  lot  of  time  and  money  to  get  set  vp  for  systens 
integration  testing  and  that  has  paid  off  well. 

-  Verification  of  software  is  done  very  well.  Few  code  errors 
now  appear.  Automating  tests  is  easy. 
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3.5  TASK  3  RESULTS:  DEVELOT4WT  CF  THE  FCS  V&V  l^THCFOLOGY 


3.5.1  Technical  Approach 

The  FTI  technical  approach  to  the  develqprient  of  the  fli^t  critical 
systens  verification  and  validation  methodology  is  based  on  a  balanced 

allocation  of  technical  skills/  proven  V&V  tools  and  techniques/  and  evolving 
software  developmental  test  methodologies.  The  iraplenentation  of  our 
methodology  will  provide  a  workstation  environment  providing  the  needed  tools 
and  techniques  for  verification  and  validation  of  flig^it  criticad  systens. 
This  approach  will  address  the  growth  in  the  use  of  software  as  the 
inplenentating  and  integrating  media  for  the  develpproent  of  hic^ily  integrated 
flight  critical  systems.  Our  overall  technical  approach  for  developing  and 
inplementing  the  methodology  is  illustrated  in  Figure  3. 5. 1-1. 

It  is  built  to  address  the  fli^t  critical  systems  software  develcpment 
tasks  and  will  provide  timely  evaluatioixs  for  each  develcpment  milestone.  Our 
approach  addresses  each  of  the  (develcpment  phases  and  breaks  out  the 

verification  and  validation  taskS/  tools,  and  technicjues  which  most 
apprcpriately  can  be  used  to  evaluate  the  develcpment  efforts  at  that  phase.  A 
large  data  base  of  analysis,  verification,  and  valiciation  tools  is  available, 
ipprcpriate  tools  will  be  chosen  to  acidress  the  V&V  recpuirements .  Those  tools 
vhich  can  be  used  (directly  to  meet  the  recpuirements  will  be  candicdates  to  be 
used  in  the  develcpment  of  a  Conputer  AicJed  Verification  And  Valiciation 
Engineering  System  (CAV^ES)  vhich  can  be  applied  to  the  current  development 

environment  phase.  For  each  tcxjl  evaluated  that  does  not  meet  specific 

criteria,  deficiencies  which  must  be  corrected  will  be  icientified  and  estimates 
of  the  effort  recpuired  to  correct  these  (deficiencies  will  be  macie. 

The  CAV^ES  will  be  hosted  in  a  workstation  environment  and  will  provide 
the  user  with  ready  acxass  to  those  recpuiratents,  design,  and  development 
details  needed  to  assess  the  state  of  (develcpment  of  PCS  software.  Mach  of  the 
early  (development  verification  activities  will  utilize  tools  hosted  in  a 
workstation  environment  and  will  provide  analysis  data  and  results  vhich  can  be 
carried  from  one  stage  to  the  next.  The  V&V  methodology  will  also  address  the 
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valicJation  test  activities  which  take  place  when  the  development  progresses  to 
the  point  where  testing  of  software  and  subsystems  utilizes  hotbench 
simulators,  simulators  requiring  mainframes  for  ccnputational  support  and 
finally  vhen  testing  moves  into  an  irohbird  test  environment. 

Review/selecticm  of  tools  and  techniques  will  be  an  ongoing  activity.  This 
activity  will  address  inpacts  on  V&V  requirements  as  future  fli^t  critical 
systons  software  designs  take  advantage  of  the  growth  in  ccnputational  pxjwer 
due  to  ccnputer  advancements  and  in  new  design  approaches  vhich  can  be  utilized 
becaxase  of  this  growth.  Increcises  in  levels  of  redundancy  for  increased 
safety,  use  of  highly  integrated  FCSAMS/PVI  to  inprove  mission  performance, 
and  use  of  self-repairing  design  techniques  in  FLCS  are  but  a  few  of  the  trends 
vhich  will  conplicate  the  verification  and  validation  of  PCS  software. 

The  proposed  V&V  methodology  will  provide  the  user  a  means  for  early 
identification  of  ambigpiities  and  errors  in  requirements  generation  and  design, 
yet  also  provide  the  means  of  assessijog  vhether  or  not  the  PCS  operates  as 
required  and  is  flight  worthy. 

The  specific  details  of  the  CAV^ES  will  be  presented  in  the  next  section. 
3.5.2  Computer  Aided  Verification  and  Validation  EnaineerinQ  System 

The  PCS  V&V  methodology  design  will  be  inplemented  in  a  workstation 
environment  denoted  as  the  Ccnputer  Aided  Verification  and  Validation 
Engineering  System  (CAV^ES) ,  see  Figure  3. 5. 2-1.  The  CAV^ES  v ill  provide 
an  environment  in  vhich  the  flight  control  engineer  or  software  engineer  can 
quickly  and  easily  access  and  ancilyze  design  information  and  software  code,  or 
generate  data  to  verify  and  validate  PCS  software.  It  will  allow  the  engineers 
to  deal  with  all  phases  of  the  development  cycle  and  tackle  the  probloii 
maintaining  a  continuity  of  requirements/design  evaluations  across  these 
phases. 

The  Computer  Aided  Verification  and  Validation  Engineering  System  will 
provide  the  following  fi’nctions:  V&V  Executive  (VEXEC) ,  Tool  Interface  Manager 
(TIM),  User  Interface  (UI),  Simulation  Conputer  Interface  (XI),  Data  Recording 
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Figure  3. 5. 2-1  Coiputer  AicJed  Verification  and  Validation 
Engineering  System  (CAV^EIS) 


System  (DRS),  Data  Base  Manager  (DBM),  Autanatic  Pilot  Functions  (APF),  and  the 
Data  Analysis  System  (DAS) . 

In  addition  to  the  CAV^ES  primary  functions,  it  also  retains  its  ovm  data 
base  and  library  of  tools.  The  data  base  is  structured  to  be  able  to  store  and 
retrieve  the  data  items  v^ch  sure  a  product  of  executing  the  verification  and 
validation  analysis  tools.  The  tool  library  consists  of  two  parts,  a  generic 
tools  set  and  an  external  tools  set.  The  generic  tool  set  is  a  group  of  V&V 
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tools  viiich  will  perform  basic  V&V  functions  on  PCS.  The  external  tools  set 
represents  user  selected  or  new  tools  vAiidi  need  to  be  interfaced  through  the 
Tool  Interface  Manager. 

The  Tool  Interface  Manager  (TIM)  selects  vdiich  interfaces  are  used,  with  the 
selected  tools  so  that  the  output  of  the  library  tool  is  put  into  a  standard 
format  acceptable  to  the  Data  Base  ^fenager.  If  a  new  tool  (external  tool)  is 
to  be  interfaced  to  CAV^ES/  the  TIM  has  an  interface  build  capability  which 
aids  the  \iser  in  building  a  functional  interface  ^rtiich  converts  data  to  a 
format  cor^istent  with  the  existing  data  base. 

The  V&V  Executive  (VEXEC)  monitors  and  coordinates  the  operation  of 
CAV^ES  functions.  The  VEXEC  execution  involves  issuing  commands  to  the  Tool 
Interface  Ifenager/  the  Simulation  Ccrputer  Interface,  the  Data  Recording 
System,  the  Data  Base  Manager,  the  Automatic  Pilot  Functions  and  the  Data 
Analysis  System.  It  receives  commands  and  sends  responses  to  the  user  via  the 
User  Interface. 

VEXEC  is  command  driven  via  the  User  Interface  (UI) .  The  user  interfaces 
with  the  VEXEC  by  means  of  commands  transformed  through  the  UI.  VEXEC  assists 
in  selection  of:  V&V  tools,  data  analysis  techniques,  and  data  bases  to  be 
used.  VEXEX3  can  eilso  assist  in  start  vp  and  shut  down  of  simulations  and 
facilities  to  be  exercised  via  the  Simulation  Computer  Interface.  It  can 
direct  the  loading  of  simulation  softweure  in  simulat:ion  computers,  and  can  also 
control  the  initicdization  of  the  data  recording  system  and  other  CAV^ES 
subsystems . 

For  V&V  static  analysis,  VEXEC  controls  the  Loading  and  execution  of  the 
selected  V&V  tools,  selected  EX^  software,  design  structures  and  code,  and  the 
data  analysis  and  output  presentations  from  the  results.  For  ctynamic  analysis, 
VEXEC  controls  the  loading  and  execution  of  flight  test  plans  and  corresponding 
test  procedUi.es/test  cases  which  comprise  than.  Execution  of  the  test 
procedures/test  cases  can  be  performed  automatically  or  single  stepped.  All 
actions  performed  by  the  VEXEC  (as  a  part  of  static  tools  execution  or  dynamic 
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execution)  and  all  user  irputs  are  recorded  in  a  test  execution  log  vAiich  is 
available  on-line  for  review. 

The  purpose  of  the  User  Interface  (UI)  is  to  provide  direct  access  to  the 
various  software  tools  functicaxalities/  \diile  relieving  the  user  of  needing 
intiinate  )aiowledge  about  the  software  tools  as  stand-alone  systems  and  adapting 
to  their  various  styles  and  syntaxes.  This  means  that  a  user  who  wants  to 
cijtain  a  time  history  plot  of  data  generated  by  a  simulation  tool,  ACSL  for 
exaitple,  does  not  need  to  know  the  particular  ccmmands  for  the  tool  package  for 
simulation  and  plotting.  However,  the  UI  does  not  confine  the  experienced  tool 
user  to  stay  within  the  UI  interface,  but  provides  a  direct  tool  mode  in  vhich 
the  user  can  execute  tool  corrmands  within  the  CAV^ES  environment  to  perform 
any  simulation  and  plotting  activity  allowed  by  the  tool.  The  UI  provides  both 
menu  driven  and  ooninand  driven  (for  the  more  ejqperienoed  user)  capabilities  to 
the  user.  The  UI  provides  an  open,  custonizable,  flexible  environment.  The  UI 
is  built  in  a  windows  environment  to  provide  quick  e3q>ansion  or  contraction  of 
backup  informatics,  aiding  in  the  verification  and  validation  process.  It  is 
also  grafdiically  oriented  in  terms  selection  of  options  and  in  presentation  of 
specification  and  design  information.  The  functionality  of  a  tool  can  be 
accessed  via  point-and-click  mouse  operations  on  icons,  menu,  and  form  driven 
scareens.  Program  structures  are  presented  by  schenatic  designs  and  network 
trees;  the  data  displays  offer  great  flexibility  in  manner  of  display  and  in 
customization. 

The  Simulation  Conputer  Interface  (SCI)  is  used  to  conmunicate  to  the 
simulation  computers.  Communication  may  take  place  over  serial  lines  to 
various  devices  and  over  ethemet  or  bus  link.  The  user  may  cpen  a  terminal 
window  for  each  of  these  connections  cind  manually  type  commands.  All  ccorvinds, 
along  with  responses,  will  be  logged  and  sent  to  the  DBM  to  be  recorded  in  a 
test  execution  log.  Other  subsystems  of  CAV^ES  may  also  send  catmands  to  the 
simulation  computers.  All  cxitmands  indicate  if  a  response  is  ejpected,  SCI 
will  then  pass  along  the  command  and  wait  for  the  response,  if  necessary. 

The  Data  Recording  Systan  (DRS)  will  be  responsible  for  lecordlna 
simulation  data  (both  real-time  arxi  non-real-time)  and  transferring  data  to 
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data  base  manager.  The  DRS  receives  its  oonmands  from  the  VEXEC.  Before  a 
test  begins,  the  VEXEC  sends  a  list  of  cctrmands  to  be  executed  (recording 
script) .  The  start  test  signal  tells  the  DRS  to  begin  executing  the  recording 
script.  The  abort  signal  tells  the  DRS  to  stop  recording  and  ignore  any 
recorded  data.  The  real-time  simulation  recording  takes  place  via  a  link  (bus 
link  or  ethemet)  connected  to  the  simulation  conputers  via  SCI.  Proper 
synchronization  is  critical  if  a  valid  set  of  data  is  to  be  recorded. 

The  Data  Base  Manager  (DBM)  serves  two  purposes.  First,  to  create  and 
maintain  data  base  files  and  second,  to  conduct  data  transactions  for  other 
CAV^ES  subsystems.  THE  DBM  is  conposed  of  two  processes  to  serve  these 
purposes:  Vexec  Interface  and  Build  Update.  The  interface  process  conducts 
transactions  vidle  Build  Update  is  used  to  create  and  maintcdn  data  base 
files.  To  assist  in  performing  these  processes,  a  ccmmercially  available  data 
base  management  system,  UNIFY,  will  be  used.  UNIFY  uses  a  Host  Language 
Interface  (HLI)  to  operate  at  both  of  these  levels.  Briefly,  UNIFY' s  HLI  is  a 
library  of  standard  data  base  management  function  queries,  reports,  etc.  that 
can  be  called  from  standard  Higher  Order  Language  (HOL)  programming  language 
statements.  The  modules  and  functions  of  the  DBM  processes  will  therefore  be 
written  in  a  chosen  HOL. 

The  Automatic  Pilot  EXanctions  (APF)  subsystem  provides  the  c^sability  for 
the  CAV^ES  system  to  send  pilot  commands  to  the  aircraft  flight  control 
system.  The  APF  provides  the  c^iability  to  perform  initial  condition  trinmed 
(ICT)  to  specified  fli^t  conditions,  to  provide  flight  test  functions  (FTF) 
for  testing  of  performance  parameters  (e.g.,  steps,  doublets,  sixie  wave,  etc.), 
to  fly  fundamental  maneuvers,  and  landing  approaches.  VEXEC  obtains  commanded 
maneuvers  from  the  test  procedures  via  the  Data  Base  Manager.  The  APF  sends 
these  commands  to  the  simulation  conputers  and  fli^t  control  system  via  the 
SCI.  The  APF  uses  a  transportable  auto-pilot  model  vrfiich  may  be  hosted  on  the 
simulation  conputers  (mainframes)  or  in  the  CAV^ES  workstation  environment, 
dependent  on  the  par^iculau:  type  of  conmunications  link  used  between  the 
CAV^ES  and  the  simulation  conputers.  The  conmunications  link  must  be  fast 
enough  to  provide  realistic  ccnmands  and  feedback  for  the  autopilot  to  provide 
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prefer  control  inputs.  This  provides  flexibility  in  the  choice  of  this 
ccninunication  link  from  one  inplementation  to  the  next.  There  is  no  intent  to 
provide  "pilot  modeling"  in  this  module,  tut  sijrply  provide  the  capability  to 
fly  in  a  controlled  manner  and  to  provide  controlled  inputs  vtfiich  would 
normally  be  sipplied  by  a  pilot.  This  module  allows  elimination  of  the  pilot 
from  many  closed-loop  tests. 

The  Data  Analysis  System  (DAS)  is  designed  to  provide  the  validation 
engineer  the  c^>ability  to  examine  the  data  recorded  during  a  test  and  to 
perform  data  reductiai  techniques  oo  the  recorded  data.  A  separate  data  set  is 
created  for  each  test  case  executed.  The  DAS  may  be  xased  to  examine  the  data 
in  any  of  the  data  sets.  The  recorded  data  can  be  displayed  both  on  a  dynamic 
display  (bitmepped  graphics  CRT)  and  on  a  hardeepy  device.  Data  displayed 
during  simulation  execution  will  generally  be  plots  of  variables  as  a  function 
of  time  and  mode  switches.  Post  test  data  analysis  can  be  performed  providing 
performance  parameters  of  flig^it  critical  systems.  Additionally,  the  user  may 
specif  what  variables  are  to  be  di^layed  for  a  given  data  set. 

The  ^plication  of  the  QiV^ES  in  PCS  software  development  and  in  its 
verification  and  validation  is  shown  pictorially  in  Figure  3. 5. 2-2.  The 
CAV^ES  can  be  used  in  the  early  phases  of  PCS  software  develcpnent  efforts  to 
perform  PCS  anailysis  and  to  add  in  requirements  and  design  analysis  as 
discussed  in  Section  3.2.  As  the  PCS  development  progress,  CAV^ES  provides 
the  analysis  tools  and  techniques  to  verify  requirements  and  design,  to  perform 
OFP  subsystem  testing  at  the  unit,  module,  and  subsysten  levels.  Software 
tools  can  be  hosted  to  aid  the  user  in  evaluation  of  system  reliability. 
Development  of  analytical  techniques  are  beceming  available  to  aid  in 
performing  this  task.  The  fault  tree  approach  is  one  tool /technique  which  will 
be  evalxoated  to  inclxxie  in  the  tool  library.  Other  techniques  will  also  be 
reviewed. 

The  CAV^ES  provides  an  environment  in  which  the  V&V  engineer  can  use  the 
inputs  of  previous  verification  efforts  including  requirements  analysis,  design 
analysis,  and  code  analysis  to  t^uickly  generate  test  cases  for  execution  of  the 
EX^S  software.  Within  this  environment,  the  process  of  generating  test  cases 
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can  be  eased  by  providing  help  or  advisory  instructions  for  testing  of  specific 
subsystem/system  segments.  Also,  test  cases  previously  used  for  achieved 
specified  test  (±)jectives  can  be  quickly  ixilled  from  the  test  case,  data  base 
and  used  as  exanples.  These  examples  can  then  be  quickly  tailored  to  meet 
specific  design  test  specifications,  if  required.  Following  this,  CAV^ES  can 
provide  an  effective  "test  directors"  workstation  vtiidi  \jses  a  data  base  of 
proven  test  procedures  to  perform  hotbench  testing,  ctynamic  subsystem  testing 
and  finally  integrated  system  validation  testing  in  an  iroh&ird  environment. 
CAV^ES  not  only  provides  the  test  directors  control  capabilities  over  the 
testing,  but  also  svpplies  data  reduction  and  analysis  tools  to  analyze  the 
test  results.  Finally,  the  CAV^ES  can  be  used  to  sipport  flicfit  test  by 
examining  (real-time  simulation)  fli^t  scenarios  prior  to  fli'^t  test  to 
predict  fli^t  test  results.  These  same  analysis  tools  can  also  be  used  to 
reduce  flight  test  data  and  oonpare  them  to  predicted  results. 

3.5.3  CAV^ES  FCS  V&V  Capabilities 

Capabilities  to  be  inclxxied  within  the  CAV^ES  environment  will  provide 
the  flic^it  control/software  engineers  the  capability  to  assess  the  FCS  software 
design  in  terms  of  performance,  stability,  and  redundancy  management  analysis. 
It  will  provide  both  static  and  d^oiamic  code  analysis  tools  for  verification 
and  validation  of  flight  critical  systems  code.  It  will  also  provide  the 
capabilicy  to  perform  quick-look  analysis  of  generated  data.  It  will  provide 
the  meariS  to  veri^  and  validate  PCS  software  through  control  of  real-time 
simulators  driven  by  proven  test  prooedures/test  cases.  Specific  V&V 
capabili'  les  and  techniques /tools  to  be  applied  will  be  presented  below  in  an 
order  vhich  would  correlate  to  the  flight  critical  system  software  development. 

3. 5. 3.1  Aircraft  Flight  Critical  Systems  Analysis 

The  CAV^ES  will  provide  the  cap)ability  to  perform  verification  and 
validation  testing  of  FCS  software  by  evaluating  the  adequacy  of  the  control 
laws  with  respect  to  performance  and  stability  and  to  evaluate  system 
mechanization  with  respect  to  redundancy  management,  timing  and  bus  loading. 
To  perform  these  analysis,  3  general  types  of  flight  control  analysis  tools 
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will  be  used:  a  linear  analysis  and  design  tool,  a  nonlinear  simulation  tool, 
and  a  systan  level  emulator.  Figure  3. 5. 3. 1-1  depicts  these  tools.  The 
non-linear  simulation  will  be  used  to  evaluate  large  anplitude  characteristics 
of  digital  PCS  and  to  provide  state  space  models  for  linear  analysis.  The 
lineau:  ancilysis  tool  will  be  used  to  evaluate  stability  and  performance 
margins,  and  sanple  data  properties  of  the  closed-loop  control  system.  The 
system  level  emulator  will  be  utilized  to  veri^  that  the  inplenented  code 
represents  the  system  design  with  sufficient  accuracy  to  meet  system 
performance  requireroaits.  Outputs  of  the  system  level  emulator  can  be  used  to 
provide  iiputs  for  rigorous  exercising  of  operational  flight  code,  vhen 
available,  on  an  instruction  level  emulator  of  the  target  flight  conputer.  A 
more  detailed  discussion  of  each  of  these  types  of  tools  follows. 


Figure  3. 5. 3. 1-1  Performance,  Stability  &  Mechanization  Tools 
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The  control  law  analysis  most  insure  that  digital  FLCS  meets  the  flying  and 
handling  requirements  within  the  maneuvering  fli^rt  envelope  of  the  aircraft, 
as  specified  by  the  system  specification  and  the  MIL-SPBCs.  This  effort 
includes  control  law  specification  assessment,  non-linear  simolation,  and 
control  law  evaluation. 

Usage  of  the  nonlinear  simulation  and  linear  analysis  tools  for  control 
system  analysis  and  evaluation  is  d^icted  in  Figure  3.5.3. 1-2.  A  general 
purpose  non-linear  aircraft  simulaticm  program  incorporates  specific  aircraft 
characteristic  through  tjser-defined  modules  for  the  aerodynamic  forces  and 
mcments,  the  propulsion  system,  the  control  system,  etc.  It  can  be  run  to  trim 
the  aircraft  for  any  desired  flight  ccxxlition,  to  generate  linear  state  models 
for  the  trinmed  flight  condition,  and  to  generate  time  history  responses  for 
user-defined  inputs  representing  commands  or  external  disturbances. 

Commercially  available  linear  analysis  and  design  tools  provide  a 
corprehensive  interactive  control  design  and  analysis  software  language  system 
including  state-of-the-art  primitives  in  classical  and  modem  control 
synthesis,  matrix  analysis,  ctynamic  system  analysis,  parameter  estimation,  and 
graphical  presentation.  System  analysis  tools  that  axe  significant  for 
investigation  of  digital  flight  ccaitrol  systems  are  continuous  to  discrete 
transformation,  time  and  frequency  respxinses,  stability  and  performance 
rchustness  conputations,  MIL-F-8785C  analysis  repx>rt  generation  and  corputation 
of  control  requirements. 


There  are  two  characteristics  of  the  DFICS  vhich  are  criticcil  to  aircraft 
safety  of  fli^t.  These  are  the  aircraft  response  to  pilot  and  turbulence 
inputs,  and  the  ability  of  the  fli^t  control  systen  to  cope  with  deviations 
from  the  model  on  vhich  it  is  based.  Of  pvarticular  irtportance  for  this 
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Figure  3. 5. 3. 1-2  Ncjnlinear  Simulation  and  Linear  Analysis  Tools 

analysis  is  the  effect  of  discretization  on  the  stability  and  performance 
robustness  of  the  aircraft.  As  was  discovered  in  the  AFTI/F-16  DFLC  design, 
for  the  low  frequency  conponents  of  system  response  discretization  of  the 
analog  design  may  be  used.  However,  the  narrow  structural  notch  filters 
require  direct  discrete  design  in  order  to  avoid  pr<±>leTis  arising  fran 
aeroservoelasticity  and  the  warping  of  filter  characteristics.  Algorithms  for 
performance  and  stability  robustness  ccrputations  are  available  for  evaluation 
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of  Fix:s  software  design.  Table  3.5.3.1-1  outlines  a  nuitber  of  tasks  v^ch  can 
be  performed  to  evaluate  tte  stability  and  perfoimanoe  of  digital  control  laws 
using  linear  analysis/design  tools. 


T^le  3. 5. 3. 1-1  Caitrol  Law  Analysis  Tasks 


DISCRETIZATION  ANALYSIS 

•  discretization  errors 

•  equivalent  transfer  function  error 

STABILITY  ANALYSIS 

•  continuous  and  discrete  closed  loop 
eigenvalues 

•  multiloop  phase  and  gein  margin  based 
on  singular  value  analysis 

PERFORMANCE  ANALYSIS 

•  transient  response  for  pilot  Inputs 

•  tracking  performance  robustness  based  on 
singular  value  analysis 


DISTURBANCE  REJECTION  ANALYSIS 

•  transient  response  for  step  wind  gust 

•  RMS  tracking  accuracy  for  random  turbulence 

•  disturbance  rejection  robustness  based  on 
singular  value  analysis 

CONTROL  REQUIREMENTS 

•  control  surface  position  and  rate  for 
deterministic  and  stochastic  Inputs 


System  Mechanization  Analysis 

The  systen  mechanization  analysis  verifies  that  the  code  inplenents  the 
system  design  with  sufficient  accuracy  to  meet  performance  reguirennents .  To 
perform  this  analysis  and  validation,  the  requirements  contained  in  Software 
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Mechanizaticn  Documents,  together  vdth  the  aircraft  c^oiamics  vfould  be 
Inplemented  in  a  system  level  emulation  structure.  Ihe  emulation  can  then  be 
utilized  for  redundancy  managem^it  analysis,  timing  and  bus  loading  analysis. 

The  structure  of  a  typical  system  level  emulator  is  illustrated  in  Figiare 
3. 5. 3. 1-3.  Use  of  the  emulator  requires  that  the  DFLCS  requirements/design  be 
inplemented  (modeled)  and  that  the  appropriate  aircraft  dynamics  be  added. 
This  type  of  emulator  can  be  used  to  analyze  the  correctness  of  module  logic 
and  functions,  bus  loading,  timing  and  redundancy  management,  as  viell  as 
analyzing  the  operational  capabilities  of  a  system  and  its  conformance  to 
system  requirements. 
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The  executive  controls  the  simulator  and  sets  failures,  initial  values, 
aircraft  flight  conditions,  and  selects  the  type  of  analysis  to  be  performed. 
The  DFLCS  enulation  consists  of  modules  with  processing  inodes  and  coimunication 
links  of  sufficient  detail  to  acccnmodate  the  desired  ccrponent  failure 
conditions  to  be  injected  the  user. 

Typical  questions  ifAiich  are  addressed  throu^  xjse  of  an  emulator  are; 
o  Are  there  deficiencies  in  the  tfedianization  Document? 
o  Does  the  detciiled  software  design  faithfully  represent  the 
Mechanization  Document  requirements? 

o  What  h^^ens  in  the  case  of  certcdn  failures  vrfiich  ^pear  simultaneous 
to  the  software  (e.g.  latent  failures)? 
o  Is  the  design  suso^ible  to  particular  cases  of  interchannel  skew? 
o  Is  mode  transition  accomplished  smoothly  among  the  asynchronous 
channels?  (same  for  failure  detection  and  reconfiguration,  and  for 
failure  reset) . 

o  Do  all  tasks  get  serviced  by  the  executive  within  timing  requirements? 
o  Is  there  adverse  coupling  between  the  Failure  Manager  and  Control 
Laws,  e.g.,  are  failuire  latching  and  control  law  configuration  handled 
smoothly  by  the  flicfit  control  executive? 
o  What  is  the  minimum  acceptable  time  to  detect  and  isolate  various 
types  of  failures?  Is  that  time  requirement  met  by  the  design? 
o  What  cue  typiCcLL  inter-channel  differences  and  are  threshold 
selections  tolerate  of  these? 

o  Does  the  persistence-counter  design  provide  for  a  reasonable  trade  of 
resistance  to  false  alarm  for  speed  of  detection? 

3. 5. 3. 2  FCS  Softwcure  Design  Verification 

Software  design  analysis  activities  are  directed  at  verifying: 
o  the  allocation  of  systen  and  software  requirements  to  software 
ccxrponents 

o  the  adequacy  of  the  design  to  meet  the  requirements 
o  conpatibility  of  iihe  software  with  both  external  and  internal 
interfaces . 
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The  primary  adm  of  the  design  verification  is  to  confirm  that  the 
recommended  design  vdll  perform  the  function  specified  in  the  Requiranents 
Specification. 

The  approach  to  software  design  verification  is  illustrated  in  Figure 
3. 5. 3. 2-1.  The  approach  is  to  evaluate  the  system  requirements  documentation, 
progressing  throuc^  development  outputs  and  approved  baseline  design 
documentation.  Analysis  of  the  system  and  software  requirements  is  the  initial 
stQJ  of  design  verification  and  is  normally  addressed  as  part  of  the  PCS  and 
control  law  analysis  discussed  in  Section  3. 5. 3.1.  The  systen  specification, 
Part  I  design  specification,  interface  conpatibility  documents,  trade  studies, 
and  other  documents  provide  inputs  for  verification  of  the  requiranents 
allocation.  Requirements  are  analyzed  to  confirm  that:  all  functional, 
interface  and  test  requirements  are  ccnpletely  specified  in  quantitative  terms; 
requirements  are  logical,  consistent,  testable,  and  traceable;  all  data  base 
and  data  reqciirements  are  clearly  stated;  all  equations  have  been 
scientifically  verified;  aixi  timirjg  and  sizing  estimates  have  sufficient  margin 
for  growth.  Upon  reviewing  the  requirements  documentation,  and  the  FCS 
software  develc^inent  and  management  plans,  a  requirements  conpliance  checklist 
may  be  used  to  veri^  that  the  given  design  adequately  addresses  all  system 
requirements.  Exanple  requirements  corpliance  checklist  and  software  design 
conpliance  cliecklist  are  presented  in  Figure  3. 5. 3. 2-2. 

The  next  st^  in  the  verification  is  to  confirm  the  technical  adequacy  of 
the  design.  This  process  is  to  verify  that  the  total  design  has  been  expressed 
in  writing  and  that  adequate  analysis,  simulation  and  eAmluation  have  been 
performed  to  evaluate  the  design  as  to  risk,  expected  performance,  cost  and 
reliability.  The  design  should  address  performance  capabilities,  system  and 
software  architecture,  operational  sequences,  information  flow,  timing  and 
other  parameters.  Design  elements  are  analyzed  for  airbiguities, 
maintainability,  expandability  adequate  decomposition  of  design  corponents 
ensuring  a  top-down  design  and  that  testability  and  maintainability 
considerations  are  enbedded  within  its  structure. 
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Verify  Requirements  Allocation 


c  Review  Documentation 
t  Trace  Requirements 

•  Review  Trade  Studies 

•  Perform  Independent  Studies 


Documentation 

•  System  Specs 

•  CP  Dev  Specs 

•  OFLCS  Dev  & 
Management 
Plan  CPDP 

•  I  CDs 

•  CP  Product 
Specs 

«  Trade 
Studies 


-  Requirements  Evaluated  for 

•  Adequacy 

■  Completeness 

•  Traceability 

•  Accuracy 

•  Testability 


Verify  Design  Concepts 


Review  Documentation 
Perform  Algorithm  Analysis 
Analyze  Architecture 
Review  Trade  Studies 
Perform  Independent  Trade  Studio 
Analyze  Life-Cycle  Costs 


-►“Verification  of  Design  Concepts 
to  Ensure 

•  Compliance  with 
Requirements 

•  Feasibility 

•  Practicality 

•  Testability 


Verity  Component  Compatibility 


•  Review  Documentation 

•  Review  Trade  Studies 

•  Perform  Independent  Trade  Studies 

•  Analyze  Hardware/Software/  Pi  lot 
■  Interaction 

■  Analyze  Communication 


■Verification 
of  Interfaces 
to  Ensure 


Compliance  with 

Requirements 

Feasibility 

Practicality 

Testability 


Figure  3. 5.. 3. 2-1  /^roach  for  Performing  Software  Design  Analysis 
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Particular  attention  is  directed  to  design  strategies  v4u<±i  involve 
external  interfaces  such  as  the  hardware/software/pilot  interaction.  The 
design  is  examined  to  ensure  that  all  external  interface  requirements  have  been 
addressed. 

As  can  be  discerned  from  the  description  of  the  design  verification  tasks, 
much  of  this  effort  involves  manual  review  of  requirements  and  design 
specifications,  requiring  methodical  and  diligent  attention  in  matching 
requirements  to  design  and  in  evaluating  designs.  It  is  in  this  area  viysre  the 
explication  of  CASE  tools  discussed  in  Section  3.5  can  pro\d.de  benefits  in 
verification.  In  general,  CASE  tools  leverage  the  requirements  analysis  and 
design  specification  phases  of  the  software  development  cycle  vhile  more 
traditional  tools  are  more  applicable  in  the  software  iitplementation  phase. 
CASE  tools  can  be  ^plied  from  an  independent  viewpoint  to  evaluate 
reqijiirements/design  relations.  They  can  provide  structured  analysis  of  these 
designs/requirements  relations  to  ease  the  evaluation  process  of  the  analyst. 
CASE  tools  can  yield  tremendous  benefits  in  revealing  many  requirements  (and 
surprises)  before  inplementatior*  begins.  Seme  CASE  tools  also  provide  reverse 
engineering  capabilities  \^ch  will  take  inplemented  software  back  to  graphic 
structured  design  representations.  In  this  form,  verification  of  design  to 
design  specification  and  traoeaoility  of  requirements  to  design  can  be 
pjer formed  more  easily. 

For  this  effort,  Frontita:  will  evaluate  currently  available  tools  and 
incorporate  one  or  two  of  the  most  applicable  tools.  Teamwork,  Battlemap  and 
Traoebuilder  II  are  examples  of  CASE  tools  that  are  ^plicable  in  this  design 
verification  task. 

3. 5, 3. 3  PCS  Software  Code  Verification 

The  cbj-ctive  of  this  software  code  verification  is  to  evaluate  the  code 
for  technical  correctness,  efficiency  and  adequacy.  Tools  and  techniques  to 
aid  in  performing  software  code  verification  are  many  and  varied.  The  general 
classes  of  tools  and  techniques  were  discussed  in  Section  3.5.  The  code 
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analysis  will  utilize  tools  and  techniques  that  allow  for  test  repeatability, 
since  multiple  versions/ipdates  of  the  code  are  a  natural  part  of  any  FCS 
software  develcpment  effort.  The  code  analysis  will  enphasize  the  software 
interfaces  and  sequencing  logic.  Past  experience  shows  that  these  areas  tend 
to  be  very  error-prone.  The  code  will  be  examined  using  both  static  analysis 
and  cinnamic  executicai  analysis.  Static  analysis  provides  statisics  on  syntax, 
structxoral  relationships,  and  cross-references,  while  dynamic  execution 
analysis  will  allow  evaluation  of  actual  code  execution  results.  The  code  will 
also  be  examined  for  efficiency.  Areas  of  code  that  have  hi^  utilization  will 
be  identified  with  a  timer  analyzer  tool  and  then  will  be  further  scruntinized 
for  efficiency  in  coding  to  minimize  run  time.  Routine  and  module  size  is 
another  area  vhich  must  be  monitored.  Urplanned  trade-offs  often  occur  when 
sizing  and  timing  constraints  reach  their  limit. 

Specific  static  ancilysis  techniques  vhich  will  be  initially  inplemented 
will  be  selected  cis  a  pcut  of  the  Phase  II  effort.  However,  capabilities  to 
perform  software  sneak  analysis  and  to  perform  instruction  level  otiulator  code 
execution  arc  examples  of  static  and  cfynamic  anadysis  tools,  respectively. 
Both  provide  a  strong  code  verification  tool  base  and  are  planned  for 
iirplementation  in  CAV^ES.  A  description  of  the  use  of  these  follows. 

Software  Sneak  Analysis  (SSA) .  SSA  methodology  is  based  on  the  development 
of  topological  network  trees  vhich  provide  a  clearly  understandable  functional 
representation  of  PCS  software  performance,  and  vhich  are  useful  throughout  the 
life  of  the  FCS  software.  Software  sneak  analysis  outputs  include  a 
comprehensive,  understandable  software  network  tree  database.  Instead  of 
basing  the  develcpment  of  network  trees  strictly  on  NASA  developed  techniques 
or  other  sneak  methodologies,  FTI  builds  a  network  tree  database  from 
hierarchial  models  in  a  manner  that  clearly  reflects  a  program  function,  in  a 
format  understandable  to  hardware,  software,  and  systen  engineers.  This 
database  is  one  of  the  major  benefits  offered  by  a  software  sneak  analysis 
approach.  Initially  each  line  of  executable  source  code  is  converted  to  one  or 
more  models  as  shown  in  Figure  3. 5. 3. 3-1.  The  modelling  process  is 
hiercurchial .  For  instance,  in  a  background  executive,  the  main  program  may  be 
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represented  only  as  an  impedanoe.  In  lower  level  trees,  this  model  vd.ll  be 
ejqpanded  to  include  adl  execution  paths. 


•  EACH  LINE  OF  CODE 

IS  iiiauDED  IN  ONE  OR  noRE  nooas: 

(irpEjuicE  ) 

(REUY  COIL-COHTAaS) 

FUNCTIONAL  GROUP 

1 

OLL 

(POKER) 

(GROUND) 

(SWITCH  ) 

C  3 

V 

I 

START 

END 

Figure  3. 5. 3. 3-1  Models  for  Code  Representation 


The  next  step  in  the  SSA  process  is  to  use  pathfinding  programs  in  order  to 
fully  trace  all  possible  program  execution  paths.  The  programs  assist  in 
connecting  all  source  code  hierarchial  models  and  displaying  them  in  the 
softvrare  network  trees .  A  unique  SSA  tool — hierarchial  data  cross 
references — are  then  generated  to  assist  in  determining  sneak  data  paths  by 
helping  trace  data  flow  across  all  module  interfaces. 

Sneak  paths,  sneak  indications,  and  sneak  labels  are  determined  through 
tcpografh  identification  and  clue  application  techniques.  Srveak  timing 
problems  are  discovered  throuch  intempt  sequencing  and  operations  analysis. 
Figure  3. 5. 3. 3-2  illustrates  the  six  basic  topographic  patterns  coinon  to  all 
languages.  Eiach  netviork  tree  is  ODmposed  of  one  or  more  of  these  topographs. 
Each  pattern  (topograph)  lias  an  associated  clue  list  which  is  used  by  the 
analyst  to  alert  him  to  potential  sneak  conditions. 
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Figure  3. 5. 3. 3-2  Software  Sneak  Topogre^ihs,  Coninon  to  All 
Languages/  Alert  the  Analyst  to  Possible  Sneak  Conditions 


In  addition  to  applying  topograph  specific  clues/  a  list  of  application 
clues  are  used  to  help  determine  sneak  conditions.  An  exanple  of  sneak 
conditions  is  given  in  Figure  3. 5. 3. 3-3. 

Ihe  network  tree  modelling  technique  is  specifically  designed  to  provide 
maximum  visibility  of  code  function  so  effects  of  changes  on  overall  system 
function  can  be  easily  determined. 


81 


Figure  3. 5. 3. 3-3  Exanples  From  F-16  DFLCS  SSA  Program 

Instruction  Level  Etnulator 

To  evaluate  program  correctness,  the  FCS  software  will  be  executed  and  its 
response  to  given  stimuli  will  be  assessed  against  acceptable  limits  through 
instruction  level  emulation.  The  1750A  simulator  is  an  exanple  of  an 
instruction  level  emulator  which  will  be  eirployed  to  execute  the  software  for 
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two  levels  of  correctness  testing.  The  basic  level  of  correctness  testing  will 
exercise  all  FCS  software  in  a  routine-l^-routine  bottcm  up  fashion.  BrrfSiasis 
here  will  be  on  inputting  nominal/  limit  and  erroneous  data  into  the  routine 
and  evaluating  the  output  for  acxeptable  content  and/or  curithmetic 
correctness.  Instruction  paths  will  be  tracod/  aiding  in  the  identificration  of 
dead  code/  and  in  calculation  of  timing  estimates  for  each  routine.  As  bottcm 
\p  correctness  testing  continues/  the  interfaces  between  routines  will  be 
exercised  with  regard  to  control  and  ciata  passing.  The  second  level  of 
correctness  testing  will  be  functional.  Those  software  inplemented  functions 
iciantified  as  ctitic^cil  anc3/or  suspect  ficm  the  design  analysis  effort,  software 
sneak  analysis,  or  other  techniques  will  be  execsjted  extensively  on  the  1750A 
simulator.  The  function's  respcose  to  the  test  cJata  will  then  be  evaluated 
against  acceptable  limits.  For  both  levels  of  correctness  testing,  the  HOL 
compiler  and  linker  will  be  used  to  allow  test  drivers,  stubs  and  ciata 
extraction  hooks  to  be  linked  in  with  the  PCS  software  in  order  to  augment 
emulation  testing  capabilities. 

3.B,3.4  Stand  Alone  and  Dynamic  Subsystem  Verification  and  Validation 

The  engineering  and  formal  system  and  software  testing  utilizes  a  bottcm-ip 
philosophy.  Testing  starts  with  a  lowest  level  code  (unit  code)  axxl  progresses 
upward  to  system-level  testing.  Verification  of  results  is  performed  at  each 
level  before  progressing  to  the  next  level.  Test  plans  and  procedures  are 
prepared  for  each  level  of  formal  testix>g  and  test  results  are  dcxomented. 
Problems  or  discrepancdes  cietected  during  any  phase  of  testing  is  documented, 
investigated,  and  acted  upon. 

The  stand  cilone  verification  and  validation  is  the  first  level  of  testing 
involving  actual  flight  hardware.  The  objective  of  this  testing  is  to  insure 
that  the  Operational  Flight  Program  {(XV)  is  funcrtioncdly  correct.  Icteally, 
this  testing  should  be  automated  to  the  naximum  extent  possible  to  allow  for 
rapid  retesting  when  future  updates  are  made.  The  testing  verifies 
recjuirements  specified  in  Software  Recjuirements  Specification,  provicies 
cpen-locp  functional  evaluation,  and  supports  cc*iputer  software  configuration/ 
hcuxiware  integration  testing.  This  test  environment  uses  engineering  test 
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stations  containing  models  to  drive  different  si±>systenis  containing  flyable 
hardware  tests.  Test  files  provided  by  CAV^ES  are  \ised  to  drive  tests.  The 
resxd-ts  of  these  tests  are  then  oonpared  to  predicted  resvilts  and  test  reports 
are  autctnatically  generated. 

The  next  step  in  the  V&V  testing  is  to  integrate  individual  "subsystem" 
tests  to  provide  dynamic  rather  than  static  inputs  between  tested  subsystems. 
This  can  be  done  to  different  levels  as  illustrated  in  Figure  3.5. 3. 4-1. 
Flyable  subsystems  are  incrementally  integrated  via  engineering  test  stations. 
This  type  of  testing  provides  added  testability  of  integrated  subsystons  and 
reduced  hotbench  or  iron-bird  simulation  requirements.  CAV^ES  will  be  used 
to  communicate  to  simulation  oonputers  and  subsystems  to  drive  individual  and 
integrated  subsystem  tests.  The  Kbneuvering-simulation  Validation  Automation 
(^WA)  system  currently  under  development  by  FTI  for  SAAB-SCANIA  will  provide 
just  this  type  of  capability.  The  design  structiore  of  CAV^ES  can  incorporate 
many  of  the  functions  in  the  ^WA  system. 

3. 5. 3. 5  Integrated  System  Verification  aixl  Validation  (ISW) 

Integrated  system  verification  and  validation  testing  provides  a  closed 
loop  test  environment  that  simulates  the  ctynamic  behavior  of  the  air  vehicle. 
Actual  fli^t  hardware  is  used  where  practical  to  verify  closed  loop  dynamic 
responses.  This  test  environment  is  used  to  perform  pilot-in-the-loop  handling 
qualities  evaluation,  pilot  vehicle  interface,  and  failure  modes  and  effects 
tests.  This  test  configuration  is  represented  in  Figure  3.5. 3.5-1.  The 
QW^ES  will  be  used  to  pra/ide  a  test  directors  work  station  environment  for 
conduct  of  these  types  of  tests. 
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FIGURE  3.5.3.4-1  SUBSYSTEM  VERIFICATION  &  VALIDATION 


The  entire  system  test  and  evaliaation  process  is  represented  in  Figure 
3. 5. 3. 5-2.  This  includes  the  FCS  and  software  along  with  airplane  subsystem 
tests.  The  culmination  of  all  these  tests  is  fli^it  testing. 

In  summary/  the  CAV^ES  is  an  integrated  V&V  tool  set  to  be  used 
throuc^out  PCS  developnent  phases.  The  specific  tools  and  techniques  discussed 
for  inplementation  will  provide  the  user  the  capability  to  analyze  the  PCS 
software  design  and  code,  to  allow  him  to  easily  generate  test  cases#  and  to 
execute  these  test  cases  in  various  test  environraents.  It  provides  the 
capability  for  performing  reliability  analysis  through  user  selected 
techniques.  The  user  is  able  to  call  on  analysis  data  generated  from  different 
V&V  analysis  phases  as  irput  to  his  current  task.  It  is  usable  by  developers, 
V&V  organizations,  and  research  grorps.  Its  workstation  provides  the  user  with 
a  powerful  conputational  environment  which  can  utilize  many  of  the  more 
sophisticated  PCS  design  and  analysis  tools.  CAV^ES  provides  a  CASE  type 
working  envirorment,  user  friendly  interfaces,  and  provides  hooks  for  addition 
of  future  tools.  It  provides  means  for  interfacing  and  driving  real-time 
simulatiais  and  offers  an  environment  to  approach  automatic  generation  of  test 
cases  vAiile  also  aiding  automation  of  real-time  testing.  CAV^ES  is  an 
evolutionary  system  which  accommodates  new  tools  to  meet  the  growing  user 
requirements  in  verification  and  validation  of  flight  critical  systems 
software. 
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Figure  3.5.3 


.5-1  Integrated  System  Verification  and  Validation 
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SUBSYSTEM  HOTBENCH 

OFP  STANDALONE  SIMULATOR 

V  4  V  TESTING  CHECKOUT 


88 


FIGURE  3.5.3.5-2  SYSTEM  TEST  AND  EVALUATION 


i^ijpendix 


Data  Gathering  Checklists 
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OTTF.STIONNAIRE  TOPICS 


TYPES  OF  SOFTWARE  DEVELOPED 
DEVELOPMENT  PROCEDURES /METHODOLOGY 
MANAGEMENT  OF  DEVELOPMENT  EFFORT 

FLIGHT  CRITICAL  SYSTEMS  DEVELOPED  BY  THESE  METHODS 

DEVELOPMENT  LANGUAGES 

SOFTWARE  TOOLS  AND  TECHNIQUES 

STANDARDS,  PRACTICES  AND  GUIDELINES 

VERIFICATION  AND  VALIDATION  TECHNIQUES 

QUALITY  ASSURANCE  TECHNIQUES 

IMPACT  OF  GOVERNMENT  STANDARDS 

GENERAL  FACILITY  DATA 

FLIGHT  CONTROL  SYSTEMS  REQUIREMENTS  ISSUES 
GENERAL  PROBLEM  AREAS  IN  DEVELOPMENT 
SYSTEM  DEVELOPMENT  ERRORS  AND  STATISTICS 
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TYPES  OF  SOFTWARE  DEVELOPED 

Flight  Control 
Avionics  Architecture 
Engine  Control 
Command,  Control,  Comm 
Analytical  Models 
Simulation 


DEVELOPMENT  PROCEDURES /METHODOLOGY 

Waterfall 

Spiral 

Prototyping 

Hierarchical  Phase  Model 
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Systen  Requimnents 
Systems  analysis  and  design 
Software  analysis  and  design 
Coding  and  dieckout 
Software  integration  testing 
Software/System  qualification 
Software  Maintenance 


Stage 

Reviews 

Tests 

1, 

Requirements 

Systems  require¬ 
ments  review 

-- 

2. 

System  design 

Software  concept 

— 

3, 

Software  design 

Preliminary  design 
review 

— 

4. 

Coding 

-- 

Module  tests 

5. 

Integration 

Critical  design 
review 

Integration  tests 

6. 

Qualification 

Qualification 

Validation  tests 

Audit  or  functional 

on  operational 

Configuration  audit 

hardware 

7. 

Installation 

Physical  configura¬ 
tion  audit  and 

Validation  tests 

Formal  qualification 
review 

On  iron-bird 

simulation 

L±_ 

Maintenance 

Change  reviews 

Re- validation  tests 
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MANAGEMENT  OF  DEVELOPMENT  EFFORT 


o  Background  of  software  designers:  control  engineers?  software  engineers? 

o  What  aspects  of  software  project  management  functions  are  used: 
Management 

-  Project  Planning 

-  Project  Control 

-  Project  Communications 
Documentation 

-  Engineering  Documentation  (2167,  PDL,  etc.) 

-  Formal  Management  Documentation  (2167,  SDP,  S/W  Dev  Plan, CM  Plan, 

QA  Plan 

-  Informal  Documentation  (reports,  guidance/policy  papers,  minutes, 

etc. ) 

Configuration  Management 

-  Baseline  Identification  (Tech  descrip  of  all  S/W  items) 

-  Control  &  Tracking  of  S/W  Access  and  Change 

-  Control  of  S/W  releases 


FLIGHT  CRITICAL  SYSTEMS  DEVELOPED  BY  THESE  METHODS 

F-16  DFCS 

F-15  E 

AFTI/F-16 

F-15  STOL 

X-29 

F-18 

PAGUS 

X-31 

F-117 


DEVELOPMENT  LANGUAGES 


Ada 

Jovial  J73  (MIL-STD  1879B) 

Assembly 

CMS-2 
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SOFTWARE  TOOLS  AND  TECHNIQUES 


Proprietary  Tools 


Flight  Control  Analysis  Tools 

-  Matrix-X 

-  CNTRL-C 

-  MATLAB 


CASE  tools 


TECHNIOOp 

Abstractions  and  hierarchies  to  reduce  oonplexity:  abstractions  such 
t  rees  are  used  to  make  the  design  sinple  and  clearly  defined. 

Checkout  (debug)  testing;  function/inodule  testing  before  integration 
Constructive  design  approaches:  (eg.  formal  design  language) 

Crlticcil  design  Review;  orod  demonstration  of  detailed  design 
Dat  .  flow  diagram,  structure  chart;  used  in  preliminary  design 
Descriptions  or  documentaticxi 

Design  guidelines,  test  guidelines,  &  coding  guidelines 
Design  standards,  coding  standards 

Functional  capabilities  list:  module  descripticn  of  functions  to  perform 

Integration  testing:  code  test  after  modules  are  assenbled;  I/O  struct. 

Organization  as  finite  automata 

(Xialification  audit 

Singularities  and  extremes  testing 

Syntolic  executicai:  performed  cn  ^aecial  functions  such  as  mocte  logic 
Systems  cxncept  review;  oral  cJemo  of  initial  concepts,  trade-offs,  etc. 
Valiciation  testing:  final  demo  in  simulation  environroent 
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TOOLS 

Accuracy  analyzer  -  analyzes  numerical  calculations  for  red  accuracy 

Circular  reference  (Checker  -  modules  calling  each  other 

CAE 

Code  conparator  -  differencing  between  versions 

Cross-reference  checker  -  calling  of  modules;  external  variables  called 
Data  bases  analyzer  -  module  accesses  to  data  bases;  unused  elements 
Data  Flow  Pathing  -  trace  execution  sequence  for  variable  (s)  in  flow 
Documentation  &  constructions  -  Auto  documentation;  Consistent  data  pool 
Editor  -  Analyze/extract  information/relationships  from  source  programs 
Emulations  -  System  level  model  generated  from  requirements,  not  design 
Flow  charter  -  show  logical  construction  of  program 
Formal  Languages  -  Structured  and  rules - 

Interface  Checker  -  Check  Fange,  limits,  scaling  of  variables 
Module  Invocation  Tree  -  Establishes  call  hierarcSiy  vdth  system 
Program  flow  analyzer  -  statistics  on  usage;  estimate  execution  time 
Set/Use  checker  -  Checks  for  variables:  set,  not  xjsed;  &  used  before  set 
Simulations  -  Test  characteristics,  algorithms,  functions,  performance 
Sneak-Path  Analyzer  -  Looks  for  unexpected  paths 

Syntolic  evaluator  -  reconstructs  equations  relating  output  to  irput 
Test  data  generator  -  produces  test  cases  to  exercise  the  system 
Test  driver  -  controls  the  execution  of  a  program 

Test  execution  monitor  -  collects  data  and  ccnpares  to  expected  results 
Test  record  generator  -  analyzes,  reduces,  and  formats  results 
Theorem  prover  -  axioms  used  prove  assertions  stated  for  a  path 
Timing  analyzer  -  monitors/reoords  run  time  of  functions  and  routines 
Units  oaasistency  checker  -  variable  expressions  (units)  checked, 
unreachable  code  detector  -  looks  for  code  which  cannot  be  executed 


Specific  Static  Tools 

General 

Static  Tools 

Dynamic  Tools 

Circular  reference  checker 

Accuracy  analyzer 

Simulations 

Code  comparator 

Assembly  code  verifier 

•  Computer 

Consistency  checker 

Assertion  checker 

•  Hybrid 

•  Test  bed  (iron  bird) 

Cross  -  reference  checker 

Documentation  and 

•  Monte  Carlo 

Dnta  base  analyzer 

construction  sysli’ins 

Test  data  generator 

Formal  languages  with 

Test  driver 

!•  loiv  cliarlcr 

syntax  analyzers 

Interface  checker 

•  Hequirements 

'l  est  execution  monitor 

Program  flow  analyzer 

•  Specifications 

Test  record  generator 

Scl/use  checker 

•  Program  design 

Timing  analyzer 

•  Program  code 

Standards  checker 

Snr.'ik  pt'itli  ;nKily7fr 

Pnitt;  consistency  cliecker 

Symbolic  evaluator 

t  nrcacti.ihlc  code  di’tecior 

'I  heorem  pi  ovi  i 

\  rnlu  ‘t  n'J-  «  lO’i 
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STANDARDS,  PRACTICES  AND  GUIDELINES 

-  Design  standards 

-  Coding  Standards 

-  Documentation  Standards 

-  Engineering  Development  Standards 


VERIFICATION  AND  VALIDATION  TECHNIOtTRS 

-  Review  &  Walkthrough 

-  Units  Consistency  Checks 

-  Data  Flow  Charts 

-  Interface  Format  Checks 

-  Decision  Tables 

-  Set /Use  Checks 

-  Type  Consistency  Checks 

-  Timing  Test/Analysis 

-  Numerical  Accuracy/Precision  Analysis/Test 

-  Symbolic  Evaluation 

-  Testing 

-  Control  Flow  Diagram 

-  Petri  Nets 

-  Correctness  Proof 

-  Scaling  Analysis 

-  Table  of  Events 

-  Simulation 
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QUALITY  ASSURANCE  TECHNIQUES 

-  Software  Inspections 

-  Software  Reviews 

-  Software  Audits 

-  Document  Reviewed  for 

—  Consistency 
—  Traceability 
—  Clarity,  readability 
—  Structure 

—  Completeness  with  respect  to  H/W  &  S/W 

-  Standards,  Practices,  and  Conventions 

-  Configuration  Management 

-  Quality  Factors  and  Criteria 

-  Code  Control  and  Media  Control 

-  S/W  Quality  Assurance  Standards 


IMPACT  QF  GQVERNMENT  STANDARDS 

MIL-STD-2167A 

MIL-STD-2168 

MIL-STD-F-9490D 

MIL-STD-483 

MIL-STD-490 

DI-S-30567A  (CPDP) 

MIL-STD-1521A 
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GENERAL  FACILITY  DATA 


General  Data; 

Corcputer  Data : 

Cockpits : 

Graphics  Data: 

Special  Equipment: 

Sof'tvare  Library: 

Facility  Problems: 

Facility  Plans  and  Goals: 
Processor  Requirements: 

S/U  Requirements: 

Contractor  S/W: 


Frontier  Technology,  Inc. 


Company,  laboratory  name.  address,  key 
contacts,  etc. 

Host  processors,  distributed  processors,  I/F 
diagrams,  size  of  memory. 

Type  (i.e.,  fighter,  transport,  generic) 
displays  (analog,  CRT,  etc.)  controls  (i.e., 
stick,  wheel,  force -feel,  etc.). 

Resolutions,  colors,  update  rates,  features, 
etc. 

As  available  from  contractors  containinz 
data  on  special  test  support  or  software 
development  equipm*  .at. 

Program  names  and  fiinctions. 

The  contractor's  estimation  of  areas  of 
weaknesses  and  needed  enhancements  for 
flight  critical  software  test. 

New  equipment  expected,  desired,  or  in 
procurement. 

What  processors  will  be  used  to  host 
in-house  S/W  development,  memory 
requirements,  timing  data. 

Operating  systems,  languages  (HOL  and 
Assembly)  FC  Development  tools  for 
development  test  and  demo.  Emulators,  etc. 
of  all  stages  of  development  showing,  S/W 
test  and  support  requirements . 

Those  software  development,  tests  and  demo 
packages  that  are  required  to  support  flight 
critical  system  development  and  verification 
including  software  availability  from 
contractor  with  lease  -  rent-buy  cost  data. 
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FLIGHT  CONTROL  SYSTEMS  REQUIREMENTS  ISSUES 


o  Requirements.  Methods  should: 

-  Be  problem  driven  and  applications  based 

-  Handle  H/W  and  S/W  interactions  in  concurrent  mechanizations 

-  Address  complexity  and  reliability  concerns  explicitly  and 

quantitatively 

-  Effect  component  integration  during  design 

-  Provide  provision  for  quality  assurance 

-  Natural  transition  for  system  reqs  to  software  design  to  H/W-S/W 

implementation  and  integration 

o  Integrated  support  environment  is  needed. 

o  S/W  development  environment  will  need  to  be  incorporated  into  a 
system  prototype  facility. 

o  Different  levels  of  abstraction  appropriate  for  different  phase  of 
development  and  post-development 

O  MOST  IMPORTANT  REQUIREMENTS 

o  Satisfaction  of  fast  real-time  constraints 
o  High  reliability 
o  High  availability/survivcibility 

o  Supportability/ease  of  modification/ease  of  test 
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GENERAL  PROBLEM  AREAS  IN  DEVELOPMENT 


What  measures  of  fault  tolerant  design  are  used? 

What  measures  of  fault  avoidance  design  are  used? 

Structured  design  methods;  QA;  systematic  testing;  small, 
simple  modules. 

Methods  of  countering  impact  of: 

-  Increased  use  of  relaxed  static  stability 

-  Integrated  avionics  and  control  functions 

-  Other:  complex  PCS  Algorithms  for  sensor,  sensor  blending 


SYSTEM  DEVELOPMENT  ERRORS  AND  STATISTICS 

Types  of  errors 
Ease  of  finding 
Measures  of  Performance 
Gathering  of  statistics 
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(5)  software  integration  testing  and 

(6)  software/systein  qualification 
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